[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:40 dist-test-slave-vs05 krb5kdc[15810](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:40 dist-test-slave-vs05 krb5kdc[15810](info): set up 2 sockets
Sep 27 18:43:40 dist-test-slave-vs05 krb5kdc[15810](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260927 18:43:41.179656 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:41.306298 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35497
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:41.389358 15823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:41.389619 15823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:41.389657 15823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:41.390789 15823 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:41.390862 15823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:41.390885 15823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:41.390908 15823 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:41.390937 15823 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:41.392429 15823 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:35497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:41.392714 15823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:41.392940 15823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:41.398747 15829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.398758 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.398770 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:41.398795 15823 server_base.cc:1034] running on GCE node
I20260927 18:43:41.399338 15823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:41.399644 15823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:41.400833 15823 hybrid_clock.cc:648] HybridClock initialized: now 1790534621400821 us; error 44 us; skew 500 ppm
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:41.403481 15823 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:41.403931 15823 webserver.cc:460] Webserver started at http://127.15.110.254:41871/ using document root <none> and password file <none>
I20260927 18:43:41.404162 15823 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:41.404232 15823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:41.404381 15823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:41.405540 15823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "a7db4fe8bec64157826b0d78984e9cbd"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "d3788d51fdd80be957ddefad35d5e1ba"
server_key_iv: "370970ebc2b2f09a87a5aff7e13b47c3"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.405932 15823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "a7db4fe8bec64157826b0d78984e9cbd"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "d3788d51fdd80be957ddefad35d5e1ba"
server_key_iv: "370970ebc2b2f09a87a5aff7e13b47c3"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.407591 15823 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:41.408391 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.408679 15823 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:41.408783 15823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "a7db4fe8bec64157826b0d78984e9cbd"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "d3788d51fdd80be957ddefad35d5e1ba"
server_key_iv: "370970ebc2b2f09a87a5aff7e13b47c3"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.408871 15823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:41.426515 15823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:41.428507 15823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:41.428685 15823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:41.432696 15823 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:35497
I20260927 18:43:41.432740 15890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:35497 every 8 connection(s)
I20260927 18:43:41.433123 15823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:41.434084 15891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:41.436527 15891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd: Bootstrap starting.
I20260927 18:43:41.437114 15891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:41.437413 15891 log.cc:824] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:41.438139 15891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd: No bootstrap required, opened a new log
I20260927 18:43:41.439836 15891 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } }
I20260927 18:43:41.439975 15891 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:41.440023 15891 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7db4fe8bec64157826b0d78984e9cbd, State: Initialized, Role: FOLLOWER
I20260927 18:43:41.440179 15891 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } }
I20260927 18:43:41.440244 15891 raft_consensus.cc:388] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:41.440284 15891 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:41.440336 15891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:41.440832 15891 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } }
I20260927 18:43:41.440943 15891 leader_election.cc:302] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a7db4fe8bec64157826b0d78984e9cbd; no voters:
I20260927 18:43:41.441133 15891 leader_election.cc:288] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:41.441191 15894 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:41.441365 15894 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [term 1 LEADER]: Becoming Leader. State: Replica: a7db4fe8bec64157826b0d78984e9cbd, State: Running, Role: LEADER
I20260927 18:43:41.441392 15891 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:41.441457 15894 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } }
I20260927 18:43:41.441598 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 15823
I20260927 18:43:41.441702 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:41.442122 15895 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a7db4fe8bec64157826b0d78984e9cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } } }
I20260927 18:43:41.442219 15895 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:41.442269 15896 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7db4fe8bec64157826b0d78984e9cbd. Latest consensus state: current_term: 1 leader_uuid: "a7db4fe8bec64157826b0d78984e9cbd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a7db4fe8bec64157826b0d78984e9cbd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35497 } } }
I20260927 18:43:41.442374 15896 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:41.442557 15803 external_mini_cluster.cc:1428] Setting key f952a77bd7f221c37df7c5871fffcb90
I20260927 18:43:41.443127 15901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:41.443681 15901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:41.446435 15901 catalog_manager.cc:1347] Generated new cluster ID: 41e2e6dc7d8e4e12b0b49149da5df6a5
I20260927 18:43:41.446491 15901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:41.453791 15901 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:41.454296 15901 catalog_manager.cc:1504] Loading token signing keys...
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:41.459100 15901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7db4fe8bec64157826b0d78984e9cbd: Generated new TSK 0
I20260927 18:43:41.459293 15901 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:41.460556 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.443431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49864 (local address 127.15.110.254:35497)
0927 18:43:41.445799 (+ 2368us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.445806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.445827 (+ 21us) server_negotiation.cc:407] Connection header received
0927 18:43:41.445915 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.445919 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.445963 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.446057 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.448190 (+ 2133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.449978 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.451514 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.452122 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.452220 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.456468 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.456472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.456472 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.456483 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.458458 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.458988 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.458991 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.458992 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.459099 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.459421 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.459422 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.459423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.459687 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.459799 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.460157 (+ 358us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.460314 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2325,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:41.486585 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:41.567694 15917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:41.567932 15917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:41.567971 15917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:41.569126 15917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:41.569195 15917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:41.569247 15917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:41.570765 15917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:41.571035 15917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:41.571298 15917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:41.576404 15923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.576905 15922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.576926 15925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:41.577438 15917 server_base.cc:1034] running on GCE node
I20260927 18:43:41.577713 15917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:41.577965 15917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:41.579113 15917 hybrid_clock.cc:648] HybridClock initialized: now 1790534621579102 us; error 27 us; skew 500 ppm
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:41.581636 15917 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:41.582106 15917 webserver.cc:460] Webserver started at http://127.15.110.193:39419/ using document root <none> and password file <none>
I20260927 18:43:41.582369 15917 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:41.582415 15917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:41.582535 15917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:41.583662 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "5a99b0fb77124a8dae342f64aeb46d30"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "0be5e6bbbea961aeea4c8866379ad026"
server_key_iv: "c8e6a47591a23ad73a2ade9a8d1b1397"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.584022 15917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "5a99b0fb77124a8dae342f64aeb46d30"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "0be5e6bbbea961aeea4c8866379ad026"
server_key_iv: "c8e6a47591a23ad73a2ade9a8d1b1397"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.585388 15917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:43:41.586136 15932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.586344 15917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:41.586441 15917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "5a99b0fb77124a8dae342f64aeb46d30"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "0be5e6bbbea961aeea4c8866379ad026"
server_key_iv: "c8e6a47591a23ad73a2ade9a8d1b1397"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.586529 15917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:41.604393 15917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:41.606236 15917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:41.606398 15917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:41.606638 15917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:41.606954 15917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:41.607016 15917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.607059 15917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:41.607087 15917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.613337 15917 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39949
I20260927 18:43:41.613386 16045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39949 every 8 connection(s)
I20260927 18:43:41.613691 15917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:41.621613 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 15917
I20260927 18:43:41.621721 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:41.621888 15803 external_mini_cluster.cc:1428] Setting key 21cfcc9194834b84c066a24c1db0fa0c
I20260927 18:43:41.623234 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.614358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:47253 (local address 127.15.110.254:35497)
0927 18:43:41.614438 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.614440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.614697 (+ 257us) server_negotiation.cc:407] Connection header received
0927 18:43:41.615160 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.615161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.615215 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.615286 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.616234 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.616783 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.617809 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.618005 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.618100 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.620240 (+ 2140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.620242 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.620242 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.620249 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.621626 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.622117 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.622119 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.622119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.622187 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.622491 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.622492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.622492 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.622645 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.622737 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.623022 (+ 285us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.623121 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:43:41.623391 16048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.614356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35497 (local address 127.15.110.193:47253)
0927 18:43:41.614544 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:41.614587 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:41.615073 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:41.615368 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:41.615372 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:41.615939 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:41.616136 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.616141 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.616874 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.616875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.617730 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.617732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.618118 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.618119 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.618223 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.618747 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:41.618756 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:41.620156 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:41.621715 (+ 1559us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.621723 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.621726 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.622026 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.622268 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.622269 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.622269 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.622408 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.622726 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:41.622733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:41.622866 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 18:43:41.622974 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:43:41.623135 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":105,"threads_started":1}
I20260927 18:43:41.623889 16046 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35497
I20260927 18:43:41.624017 16046 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:41.624197 16046 heartbeater.cc:511] Master 127.15.110.254:35497 requested a full tablet report, sending...
I20260927 18:43:41.624755 15855 ts_manager.cc:194] Registered new tserver with Master: 5a99b0fb77124a8dae342f64aeb46d30 (127.15.110.193:39949)
I20260927 18:43:41.625370 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:47253
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:41.646998 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:41.728274 16051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:41.728471 16051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:41.728495 16051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:41.729664 16051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:41.729708 16051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:41.729735 16051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:41.731261 16051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:41.731503 16051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:41.731734 16051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:41.736703 16059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.736706 16056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.736821 16057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:41.736900 16051 server_base.cc:1034] running on GCE node
I20260927 18:43:41.737114 16051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:41.737301 16051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:41.738435 16051 hybrid_clock.cc:648] HybridClock initialized: now 1790534621738421 us; error 26 us; skew 500 ppm
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:41.740978 16051 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:41.741514 16051 webserver.cc:460] Webserver started at http://127.15.110.194:38379/ using document root <none> and password file <none>
I20260927 18:43:41.741794 16051 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:41.741865 16051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:41.742005 16051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:41.743268 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "0cbba8a84bfe4be08267fbd6b06c510b"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "777c70fa6df2db478b0264df6ebf3819"
server_key_iv: "f8e89ce768873a7fce7f93e4ffcde5fc"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.743636 16051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "0cbba8a84bfe4be08267fbd6b06c510b"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "777c70fa6df2db478b0264df6ebf3819"
server_key_iv: "f8e89ce768873a7fce7f93e4ffcde5fc"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.744949 16051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:41.745672 16066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.745857 16051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:41.745952 16051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "0cbba8a84bfe4be08267fbd6b06c510b"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "777c70fa6df2db478b0264df6ebf3819"
server_key_iv: "f8e89ce768873a7fce7f93e4ffcde5fc"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.746025 16051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:41.759583 16051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:41.761415 16051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:41.761587 16051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:41.761832 16051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:41.762152 16051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:41.762207 16051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.762243 16051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:41.762269 16051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.768361 16051 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:32955
I20260927 18:43:41.768414 16179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:32955 every 8 connection(s)
I20260927 18:43:41.768733 16051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:41.771811 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16051
I20260927 18:43:41.771930 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:41.772101 15803 external_mini_cluster.cc:1428] Setting key 5d565ad047d8f16da1284ef544951233
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:41.779166 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.769479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:45869 (local address 127.15.110.254:35497)
0927 18:43:41.769529 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.769532 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.769750 (+ 218us) server_negotiation.cc:407] Connection header received
0927 18:43:41.770225 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.770226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.770235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.770292 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.771278 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.771797 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.773234 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.773418 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.773473 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.776011 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.776013 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.776013 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.776022 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.777571 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.778035 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.778036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.778037 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.778105 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.778411 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.778412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.778412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.778546 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.778635 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.778965 (+ 330us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.779059 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:41.779409 16182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.769431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35497 (local address 127.15.110.194:45869)
0927 18:43:41.769612 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:41.769655 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:41.770139 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:41.770345 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:41.770349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:41.770933 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:41.771128 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.771133 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.771922 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.771924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.773153 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.773157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.773503 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.773504 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.773653 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.774186 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:41.774197 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:41.775904 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:41.777683 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.777690 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.777692 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.777950 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.778186 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.778187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.778187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.778311 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.778638 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:41.778645 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:41.778804 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 18:43:41.778933 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:43:41.779116 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260927 18:43:41.779913 16180 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35497
I20260927 18:43:41.780012 16180 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:41.780238 16180 heartbeater.cc:511] Master 127.15.110.254:35497 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
I20260927 18:43:41.780707 15855 ts_manager.cc:194] Registered new tserver with Master: 0cbba8a84bfe4be08267fbd6b06c510b (127.15.110.194:32955)
I20260927 18:43:41.781114 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:45869
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:41.798916 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:45731
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:41.879444 16185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:41.879647 16185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:41.879670 16185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:41.880836 16185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:41.880880 16185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:41.880941 16185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:41.882396 16185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45731
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:41.882645 16185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:41.882874 16185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:41.887850 16191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.887861 16190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:41.887858 16193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:41.888024 16185 server_base.cc:1034] running on GCE node
I20260927 18:43:41.888293 16185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:41.888620 16185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:41.889746 16185 hybrid_clock.cc:648] HybridClock initialized: now 1790534621889724 us; error 37 us; skew 500 ppm
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:41.892222 16185 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:41.892719 16185 webserver.cc:460] Webserver started at http://127.15.110.195:37675/ using document root <none> and password file <none>
I20260927 18:43:41.892974 16185 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:41.893040 16185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:41.893194 16185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:41.894224 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "56f7fa28072c4be6a547b8892f870085"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "7a66620c15cbc75fc57260707af81078"
server_key_iv: "c42829e948f7b982d1c683fa4c3c4d58"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.894593 16185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "56f7fa28072c4be6a547b8892f870085"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "7a66620c15cbc75fc57260707af81078"
server_key_iv: "c42829e948f7b982d1c683fa4c3c4d58"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.896000 16185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:41.896689 16200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.896852 16185 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:41.896950 16185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "56f7fa28072c4be6a547b8892f870085"
format_stamp: "Formatted at 2026-09-27 18:43:41 on dist-test-slave-vs05"
server_key: "7a66620c15cbc75fc57260707af81078"
server_key_iv: "c42829e948f7b982d1c683fa4c3c4d58"
server_key_version: "encryptionkey@0"
I20260927 18:43:41.897034 16185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:41.918615 16185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:41.920873 16185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:41.921033 16185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:41.921253 16185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:41.921582 16185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:41.921617 16185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.921660 16185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:41.921687 16185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:41.927834 16185 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:34739
I20260927 18:43:41.927906 16313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:34739 every 8 connection(s)
I20260927 18:43:41.928236 16185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:41.933833 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16185
I20260927 18:43:41.933914 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:41.934106 15803 external_mini_cluster.cc:1428] Setting key 504c48263fe1ed75ef584a5a50d23a52
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:41.937484 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.928976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:40331 (local address 127.15.110.254:35497)
0927 18:43:41.929022 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.929024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.929258 (+ 234us) server_negotiation.cc:407] Connection header received
0927 18:43:41.929860 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.929861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.929874 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.929931 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.930890 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.931471 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.932465 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.932658 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.932736 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.935028 (+ 2292us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.935029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.935029 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.935036 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.936308 (+ 1272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.936660 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.936661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.936661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.936709 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.936907 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.936908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.936908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.937015 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.937064 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.937337 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.937415 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:41.937628 16316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.928947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35497 (local address 127.15.110.195:40331)
0927 18:43:41.929136 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:41.929173 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:41.929750 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:41.930012 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:41.930016 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:41.930593 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:41.930787 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.930791 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.931563 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.931565 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.932358 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:41.932360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.932742 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.932743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:41.932868 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.933392 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:41.933399 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:41.934939 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:41.936377 (+ 1438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.936383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.936384 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.936598 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.936765 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:41.936766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:41.936766 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:41.936856 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:41.937073 (+ 217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:41.937077 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:41.937189 (+ 112us) client_negotiation.cc:769] Sending connection context
0927 18:43:41.937269 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:43:41.937378 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260927 18:43:41.938189 16314 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35497
I20260927 18:43:41.938289 16314 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:41.938472 16314 heartbeater.cc:511] Master 127.15.110.254:35497 requested a full tablet report, sending...
I20260927 18:43:41.938881 15855 ts_manager.cc:194] Registered new tserver with Master: 56f7fa28072c4be6a547b8892f870085 (127.15.110.195:34739)
I20260927 18:43:41.939302 15855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:40331
I20260927 18:43:41.946519 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:41.968750 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.961459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36498 (local address 127.15.110.193:39949)
0927 18:43:41.961624 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.961629 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.961638 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:41.961665 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.961667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.961680 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.961747 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.962242 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.962986 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.963738 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.964056 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.964114 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.966219 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.966226 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.966226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.966232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.967746 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.968099 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.968101 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.968101 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.968152 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.968317 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.968317 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.968318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.968435 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.968546 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.968615 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.968674 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
W20260927 18:43:41.969142 15960 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36498
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:43:41 dist-test-slave-vs05 krb5kdc[15810](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534621, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:41.990710 16321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:41.983702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36500 (local address 127.15.110.193:39949)
0927 18:43:41.983768 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:41.983770 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:41.983856 (+ 86us) server_negotiation.cc:407] Connection header received
0927 18:43:41.983956 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:41.983957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:41.983969 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:41.984040 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:41.984599 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.985102 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.985830 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:41.986040 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:41.986111 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:41.988318 (+ 2207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:41.988320 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:41.988320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:41.988326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:41.989743 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.990116 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.990117 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.990118 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.990169 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:41.990318 (+ 149us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:41.990318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:41.990319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:41.990397 (+ 78us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:41.990487 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:41.990551 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 18:43:41.990611 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:43:41.991271 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 15917
I20260927 18:43:41.997604 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16051
I20260927 18:43:42.002108 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16185
I20260927 18:43:42.006234 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 15823
2026-09-27T18:43:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1915 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:42 dist-test-slave-vs05 krb5kdc[16328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:42 dist-test-slave-vs05 krb5kdc[16328](info): set up 2 sockets
Sep 27 18:43:42 dist-test-slave-vs05 krb5kdc[16328](info): commencing operation
krb5kdc: starting...
W20260927 18:43:43.058682 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:43.178771 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41425
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:41425
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:43.257793 16341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:43.258036 16341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:43.258071 16341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:43.259202 16341 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:43.259275 16341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:43.259299 16341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:43.259320 16341 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:43.259349 16341 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:43.260797 16341 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:41425
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:43.261072 16341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:43.261296 16341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:43.266134 16341 server_base.cc:1034] running on GCE node
W20260927 18:43:43.266148 16346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.266156 16347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.266260 16349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:43.266506 16341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:43.266824 16341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:43.267985 16341 hybrid_clock.cc:648] HybridClock initialized: now 1790534623267964 us; error 41 us; skew 500 ppm
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:43.270377 16341 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:43.270789 16341 webserver.cc:460] Webserver started at http://127.15.110.254:39183/ using document root <none> and password file <none>
I20260927 18:43:43.271014 16341 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:43.271091 16341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:43.271252 16341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:43.272279 16341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "92006c5e810d4b8b8ccad5200d3eb840"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "af6008b6bf9c36026250375ae3254e95"
server_key_iv: "71e6d69c554a194bac67e6748efd008b"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.272637 16341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "92006c5e810d4b8b8ccad5200d3eb840"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "af6008b6bf9c36026250375ae3254e95"
server_key_iv: "71e6d69c554a194bac67e6748efd008b"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.274077 16341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:43.274830 16356 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.275030 16341 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:43.275131 16341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "92006c5e810d4b8b8ccad5200d3eb840"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "af6008b6bf9c36026250375ae3254e95"
server_key_iv: "71e6d69c554a194bac67e6748efd008b"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.275236 16341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:43.283936 16341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:43.285857 16341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:43.286020 16341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:43.290009 16341 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:41425
I20260927 18:43:43.290048 16408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:41425 every 8 connection(s)
I20260927 18:43:43.290575 16341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:43.291818 16409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:43.293357 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16341
I20260927 18:43:43.293509 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:43.293671 15803 external_mini_cluster.cc:1428] Setting key 854a229c95b61c28487a1d70c90f64bf
I20260927 18:43:43.294399 16409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840: Bootstrap starting.
I20260927 18:43:43.295084 16409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:43.295428 16409 log.cc:824] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:43.296253 16409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840: No bootstrap required, opened a new log
I20260927 18:43:43.298512 16409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } }
I20260927 18:43:43.298683 16409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:43.298718 16409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92006c5e810d4b8b8ccad5200d3eb840, State: Initialized, Role: FOLLOWER
I20260927 18:43:43.298842 16409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } }
I20260927 18:43:43.298925 16409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:43.298957 16409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:43.299011 16409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:43.299609 16409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } }
I20260927 18:43:43.299739 16409 leader_election.cc:302] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92006c5e810d4b8b8ccad5200d3eb840; no voters:
I20260927 18:43:43.299901 16409 leader_election.cc:288] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:43.299942 16414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:43.300098 16414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [term 1 LEADER]: Becoming Leader. State: Replica: 92006c5e810d4b8b8ccad5200d3eb840, State: Running, Role: LEADER
I20260927 18:43:43.300180 16409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:43.300196 16414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } }
I20260927 18:43:43.300530 16414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92006c5e810d4b8b8ccad5200d3eb840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } } }
I20260927 18:43:43.300616 16414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:43.300823 16414 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92006c5e810d4b8b8ccad5200d3eb840. Latest consensus state: current_term: 1 leader_uuid: "92006c5e810d4b8b8ccad5200d3eb840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92006c5e810d4b8b8ccad5200d3eb840" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41425 } } }
I20260927 18:43:43.300879 16414 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:43.301059 16420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:43.301901 16420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:43.303915 16420 catalog_manager.cc:1347] Generated new cluster ID: e23ec4e2908543ebbba5a2e73cb260b3
I20260927 18:43:43.303968 16420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:43.305729 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.294219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33200 (local address 127.15.110.254:41425)
0927 18:43:43.294404 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.294414 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.294447 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:43:43.295015 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.295024 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.295111 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.295297 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:43.296037 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.297433 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.298192 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.298796 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.298900 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.301965 (+ 3065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:43.301970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:43.301970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:43.301982 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:43.304145 (+ 2163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.304522 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.304525 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.304525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.304624 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.304830 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.304831 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.304832 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.305072 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:43.305217 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.305339 (+ 122us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.305480 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":86,"threads_started":1}
I20260927 18:43:43.323776 16420 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:43.324343 16420 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:43.327081 16420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92006c5e810d4b8b8ccad5200d3eb840: Generated new TSK 0
I20260927 18:43:43.327327 16420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:43.361591 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:43.456187 16435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:43.456449 16435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:43.456473 16435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:43.457614 16435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:43.457659 16435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:43.457684 16435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:43.459147 16435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:43.459401 16435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:43.459636 16435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:43.464561 16435 server_base.cc:1034] running on GCE node
W20260927 18:43:43.464497 16443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.464493 16440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.464568 16441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:43.464967 16435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:43.465164 16435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:43.466301 16435 hybrid_clock.cc:648] HybridClock initialized: now 1790534623466292 us; error 26 us; skew 500 ppm
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:43.468869 16435 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:43.469323 16435 webserver.cc:460] Webserver started at http://127.15.110.193:38273/ using document root <none> and password file <none>
I20260927 18:43:43.469578 16435 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:43.469650 16435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:43.469796 16435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:43.470817 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "baa0a9e94cd049f2acb13e2d4995273d"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "ae25f74a9b013acd137d4a9db8a2913a"
server_key_iv: "186b8dac7af1f9d57167e5cf7e452efb"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.471246 16435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "baa0a9e94cd049f2acb13e2d4995273d"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "ae25f74a9b013acd137d4a9db8a2913a"
server_key_iv: "186b8dac7af1f9d57167e5cf7e452efb"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.472565 16435 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:43.473306 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.473598 16435 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:43.473721 16435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "baa0a9e94cd049f2acb13e2d4995273d"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "ae25f74a9b013acd137d4a9db8a2913a"
server_key_iv: "186b8dac7af1f9d57167e5cf7e452efb"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.473829 16435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:43.487426 16435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:43.489848 16435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:43.490026 16435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:43.490273 16435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:43.490598 16435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:43.490657 16435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.490693 16435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:43.490721 16435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.496675 16435 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46501
I20260927 18:43:43.496728 16563 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46501 every 8 connection(s)
I20260927 18:43:43.497053 16435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:43.506682 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.497865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56441 (local address 127.15.110.254:41425)
0927 18:43:43.497964 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.497967 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.498145 (+ 178us) server_negotiation.cc:407] Connection header received
0927 18:43:43.499140 (+ 995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.499142 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.499154 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.499230 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:43.499876 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.500379 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.501450 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.501670 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.501760 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.504008 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:43.504010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:43.504010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:43.504016 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:43.505340 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.505766 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.505767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.505768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.505824 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.506097 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.506097 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.506098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.506205 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:43.506258 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.506533 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.506613 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260927 18:43:43.506958 16566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.497765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41425 (local address 127.15.110.193:56441)
0927 18:43:43.497994 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.498047 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.499001 (+ 954us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.499304 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.499309 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.499586 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:43.499777 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.499782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.500475 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.500476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.501357 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.501359 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.501769 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.501770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.501916 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.502450 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:43.502458 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:43.503918 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:43.505428 (+ 1510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.505435 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.505436 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.505689 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.505907 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.505908 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.505908 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.506025 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.506266 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:43.506271 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:43.506398 (+ 127us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.506485 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.506652 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":105,"threads_started":1}
I20260927 18:43:43.507153 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16435
I20260927 18:43:43.507277 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:43.507444 15803 external_mini_cluster.cc:1428] Setting key 840fdd60b12b10e7395760b79288bb10
I20260927 18:43:43.507547 16564 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41425
I20260927 18:43:43.507651 16564 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:43.507830 16564 heartbeater.cc:511] Master 127.15.110.254:41425 requested a full tablet report, sending...
I20260927 18:43:43.508348 16373 ts_manager.cc:194] Registered new tserver with Master: baa0a9e94cd049f2acb13e2d4995273d (127.15.110.193:46501)
I20260927 18:43:43.508906 16373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:56441
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:43.532428 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:43.615499 16569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:43.615701 16569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:43.615725 16569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:43.616932 16569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:43.616971 16569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:43.616997 16569 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:43.618534 16569 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:43.618777 16569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:43.618996 16569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:43.624169 16577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.624213 16575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.624171 16574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:43.624198 16569 server_base.cc:1034] running on GCE node
I20260927 18:43:43.624722 16569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:43.624949 16569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:43.626093 16569 hybrid_clock.cc:648] HybridClock initialized: now 1790534623626087 us; error 31 us; skew 500 ppm
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:43.628595 16569 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:43.629055 16569 webserver.cc:460] Webserver started at http://127.15.110.194:34333/ using document root <none> and password file <none>
I20260927 18:43:43.629290 16569 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:43.629357 16569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:43.629508 16569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:43.630517 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "7c417bc0d226442ab7ed017fe0b41262"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "c109b9cdfc6e6b56da428684ee036023"
server_key_iv: "a272e121ddabdaee9c593ef3474f9539"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.630887 16569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "7c417bc0d226442ab7ed017fe0b41262"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "c109b9cdfc6e6b56da428684ee036023"
server_key_iv: "a272e121ddabdaee9c593ef3474f9539"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.632299 16569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:43.633030 16584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.633205 16569 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:43.633299 16569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "7c417bc0d226442ab7ed017fe0b41262"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "c109b9cdfc6e6b56da428684ee036023"
server_key_iv: "a272e121ddabdaee9c593ef3474f9539"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.633383 16569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:43.651366 16569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:43.653236 16569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:43.653396 16569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:43.653633 16569 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:43.653935 16569 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:43.653988 16569 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.654028 16569 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:43.654055 16569 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.659726 16569 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:43557
I20260927 18:43:43.659770 16697 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:43557 every 8 connection(s)
I20260927 18:43:43.660080 16569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:43.667371 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16569
I20260927 18:43:43.667469 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:43.667654 15803 external_mini_cluster.cc:1428] Setting key eb2393e7d644417cf068acaec4294a09
I20260927 18:43:43.669946 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.660825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:37979 (local address 127.15.110.254:41425)
0927 18:43:43.660916 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.660919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.661125 (+ 206us) server_negotiation.cc:407] Connection header received
0927 18:43:43.662047 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.662048 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.662058 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.662123 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:43.662698 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.663236 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.664401 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.664623 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.664700 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.666828 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:43.666830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:43.666830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:43.666836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:43.668570 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.668948 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.668950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.668951 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.669013 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.669242 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.669243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.669243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.669389 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:43.669466 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.669774 (+ 308us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.669856 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 18:43:43.670166 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.660780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41425 (local address 127.15.110.194:37979)
0927 18:43:43.660992 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.661033 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.661960 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.662179 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.662182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.662444 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:43.662637 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.662642 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.663323 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.663324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.664313 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.664317 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.664739 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.664740 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.664849 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.665327 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:43.665336 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:43.666738 (+ 1402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:43.668652 (+ 1914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.668657 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.668659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.668874 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.669086 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.669086 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.669087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.669180 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.669468 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:43.669472 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:43.669622 (+ 150us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.669751 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.669909 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":135,"threads_started":1}
I20260927 18:43:43.670723 16698 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41425
I20260927 18:43:43.670828 16698 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:43.671020 16698 heartbeater.cc:511] Master 127.15.110.254:41425 requested a full tablet report, sending...
I20260927 18:43:43.671469 16373 ts_manager.cc:194] Registered new tserver with Master: 7c417bc0d226442ab7ed017fe0b41262 (127.15.110.194:43557)
I20260927 18:43:43.672016 16373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:37979
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:43.694217 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:46405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:43.774597 16703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:43.774837 16703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:43.774873 16703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:43.776051 16703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:43.776119 16703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:43.776166 16703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:43.777801 16703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41425
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:43.778046 16703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:43.778270 16703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:43.783360 16708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.783360 16711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:43.783387 16709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:43.783604 16703 server_base.cc:1034] running on GCE node
I20260927 18:43:43.783831 16703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:43.784086 16703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:43.785255 16703 hybrid_clock.cc:648] HybridClock initialized: now 1790534623785238 us; error 26 us; skew 500 ppm
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:43.787827 16703 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:43.788264 16703 webserver.cc:460] Webserver started at http://127.15.110.195:42033/ using document root <none> and password file <none>
I20260927 18:43:43.788498 16703 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:43.788552 16703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:43.788646 16703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:43.789611 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "263484b697c443b38105ecde6088a956"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "6c7473dfa66f1b2a7de24818eefcc96a"
server_key_iv: "3ae49631cf3bc9d522a75a8f768d0bdd"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.789903 16703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "263484b697c443b38105ecde6088a956"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "6c7473dfa66f1b2a7de24818eefcc96a"
server_key_iv: "3ae49631cf3bc9d522a75a8f768d0bdd"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.791090 16703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:43.791832 16718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.791999 16703 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:43.792074 16703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "263484b697c443b38105ecde6088a956"
format_stamp: "Formatted at 2026-09-27 18:43:43 on dist-test-slave-vs05"
server_key: "6c7473dfa66f1b2a7de24818eefcc96a"
server_key_iv: "3ae49631cf3bc9d522a75a8f768d0bdd"
server_key_version: "encryptionkey@0"
I20260927 18:43:43.792126 16703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:43.812180 16703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:43.814523 16703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:43.814713 16703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:43.814961 16703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:43.815357 16703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:43.815407 16703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.815443 16703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:43.815466 16703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:43.822832 16703 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:35597
I20260927 18:43:43.822878 16831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:35597 every 8 connection(s)
I20260927 18:43:43.823266 16703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:43.830873 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16703
I20260927 18:43:43.831033 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:43.831219 15803 external_mini_cluster.cc:1428] Setting key 465e59f58c45310057c86232c4d6e340
Sep 27 18:43:43 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:43.835386 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.824166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:59199 (local address 127.15.110.254:41425)
0927 18:43:43.824242 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.824245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.824484 (+ 239us) server_negotiation.cc:407] Connection header received
0927 18:43:43.825884 (+ 1400us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.825886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.825896 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.825954 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:43.826857 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.827549 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.829281 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.829466 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.829524 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.832407 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:43.832408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:43.832408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:43.832415 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:43.833864 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.834370 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.834371 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.834371 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.834419 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.834686 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.834687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.834687 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.834783 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:43.834832 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.835249 (+ 417us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.835323 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:43:43.835664 16834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.824145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41425 (local address 127.15.110.195:59199)
0927 18:43:43.824323 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.824377 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.825766 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.826047 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.826052 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.826461 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:43.826756 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.826762 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.827669 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.827670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.829184 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.829188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.829550 (+ 362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.829551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.829712 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.830238 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:43.830249 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:43.832312 (+ 2063us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:43.833937 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.833944 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.833947 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.834290 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.834477 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:43.834478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:43.834478 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:43.834622 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:43.834848 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:43.834854 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:43.835046 (+ 192us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.835198 (+ 152us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.835386 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260927 18:43:43.836102 16832 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41425
I20260927 18:43:43.836196 16832 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:43.836402 16832 heartbeater.cc:511] Master 127.15.110.254:41425 requested a full tablet report, sending...
I20260927 18:43:43.836853 16373 ts_manager.cc:194] Registered new tserver with Master: 263484b697c443b38105ecde6088a956 (127.15.110.195:35597)
I20260927 18:43:43.837352 16373 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:59199
I20260927 18:43:43.843050 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:43.851655 16412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.844684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33210 (local address 127.15.110.254:41425)
0927 18:43:43.844734 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.844736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.844840 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:43:43.844872 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.844873 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.844886 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.844961 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:43.845459 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.846333 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.847028 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.847414 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.847496 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.848694 (+ 1198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:43.848695 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:43.848695 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:43.848703 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:43.850394 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.850742 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.850743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.850743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.850810 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:43.850989 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:43.850990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:43.850991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:43.851238 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:43.851342 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.851430 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.851518 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:43.853854 16373 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33210:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:43.855463 16373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:43.862859 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.858018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.15.110.195:35597)
0927 18:43:43.858217 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.858223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.858595 (+ 372us) server_negotiation.cc:407] Connection header received
0927 18:43:43.858637 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.858639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.858655 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.858822 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:43.859023 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.860064 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.861624 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.862347 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.862417 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.862470 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.862719 (+ 249us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.862767 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":95,"threads_started":1}
I20260927 18:43:43.863330 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.857849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35597 (local address 127.0.0.1:43070)
0927 18:43:43.858467 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.858478 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.858525 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.858904 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.858905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.858906 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:43.858950 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.858951 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.860152 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.860154 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.861525 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.861528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.862528 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.862529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.862610 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.862622 (+ 12us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.863196 (+ 574us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.863258 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":56,"thread_start_us":56,"threads_started":1}
I20260927 18:43:43.863854 16844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.857849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46501 (local address 127.0.0.1:51322)
0927 18:43:43.858042 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.858100 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.858171 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.858560 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.858564 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.858566 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:43.858639 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.858641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.860075 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.860076 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.861959 (+ 1883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.861964 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.863558 (+ 1594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.863560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.863642 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.863647 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.863704 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.863756 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":53,"thread_start_us":68,"threads_started":1}
I20260927 18:43:43.864195 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.858192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51322 (local address 127.15.110.193:46501)
0927 18:43:43.858371 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.858375 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.858386 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:43.858425 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.858427 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.858445 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.858719 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:43.858801 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.859969 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.862089 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.863376 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.863465 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.863523 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.864036 (+ 513us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.864089 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":94,"threads_started":1}
I20260927 18:43:43.864265 16750 tablet_service.cc:1467] Processing CreateTablet for tablet aa58a3d651b2468cbd12a383ec2b6821 (DEFAULT_TABLE table=test-table [id=8e23e2eb17184ca0b285da68f243d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:43.864622 16843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.858055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:43557 (local address 127.0.0.1:33030)
0927 18:43:43.860176 (+ 2121us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:43.860186 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:43.860229 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:43.860504 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:43.860505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:43.860506 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:43.860547 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.860549 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.861920 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.861921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.863096 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:43.863099 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.864177 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.864178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:43.864236 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.864240 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:43.864496 (+ 256us) client_negotiation.cc:240] Negotiation successful
0927 18:43:43.864537 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2108,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260927 18:43:43.864643 16750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa58a3d651b2468cbd12a383ec2b6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:43.864979 16848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:43.858491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.15.110.194:43557)
0927 18:43:43.860341 (+ 1850us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:43.860345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:43.860354 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:43.860393 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:43.860395 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:43.860410 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:43.860625 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:43.860701 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.861817 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.863216 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:43.864080 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:43.864776 (+ 696us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:43.864834 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:43.864859 (+ 25us) server_negotiation.cc:299] Negotiation successful
0927 18:43:43.864897 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1814,"thread_start_us":109,"threads_started":1}
I20260927 18:43:43.864923 16498 tablet_service.cc:1467] Processing CreateTablet for tablet aa58a3d651b2468cbd12a383ec2b6821 (DEFAULT_TABLE table=test-table [id=8e23e2eb17184ca0b285da68f243d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:43.865262 16498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa58a3d651b2468cbd12a383ec2b6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:43.865532 16632 tablet_service.cc:1467] Processing CreateTablet for tablet aa58a3d651b2468cbd12a383ec2b6821 (DEFAULT_TABLE table=test-table [id=8e23e2eb17184ca0b285da68f243d747]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:43.865828 16632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aa58a3d651b2468cbd12a383ec2b6821. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:43.869021 16850 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Bootstrap starting.
I20260927 18:43:43.869594 16849 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Bootstrap starting.
I20260927 18:43:43.869796 16850 tablet_bootstrap.cc:654] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:43.870285 16849 tablet_bootstrap.cc:654] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:43.870429 16850 log.cc:824] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:43.870818 16849 log.cc:824] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:43.871335 16850 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: No bootstrap required, opened a new log
I20260927 18:43:43.871414 16850 ts_tablet_manager.cc:1397] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:43:43.871666 16849 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: No bootstrap required, opened a new log
I20260927 18:43:43.871755 16849 ts_tablet_manager.cc:1397] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:43:43.873324 16849 raft_consensus.cc:348] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.873495 16849 raft_consensus.cc:374] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:43.873539 16849 raft_consensus.cc:729] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 263484b697c443b38105ecde6088a956, State: Initialized, Role: FOLLOWER
I20260927 18:43:43.873662 16849 consensus_queue.cc:260] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.873955 16849 ts_tablet_manager.cc:1428] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:43:43.874095 16832 heartbeater.cc:503] Master 127.15.110.254:41425 was elected leader, sending a full tablet report...
I20260927 18:43:43.874083 16850 raft_consensus.cc:348] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.874281 16850 raft_consensus.cc:374] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:43.874424 16850 raft_consensus.cc:729] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c417bc0d226442ab7ed017fe0b41262, State: Initialized, Role: FOLLOWER
I20260927 18:43:43.874603 16850 consensus_queue.cc:260] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.874954 16850 ts_tablet_manager.cc:1428] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:43.874998 16698 heartbeater.cc:503] Master 127.15.110.254:41425 was elected leader, sending a full tablet report...
I20260927 18:43:43.875373 16851 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Bootstrap starting.
I20260927 18:43:43.876333 16851 tablet_bootstrap.cc:654] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:43.876631 16851 log.cc:824] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:43.879594 16851 tablet_bootstrap.cc:492] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: No bootstrap required, opened a new log
I20260927 18:43:43.879678 16851 ts_tablet_manager.cc:1397] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 18:43:43.882320 16851 raft_consensus.cc:348] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.882555 16851 raft_consensus.cc:374] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:43.882596 16851 raft_consensus.cc:729] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa0a9e94cd049f2acb13e2d4995273d, State: Initialized, Role: FOLLOWER
I20260927 18:43:43.882719 16851 consensus_queue.cc:260] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:43.883036 16851 ts_tablet_manager.cc:1428] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:43:43.883039 16564 heartbeater.cc:503] Master 127.15.110.254:41425 was elected leader, sending a full tablet report...
W20260927 18:43:43.910686 16699 tablet.cc:2369] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:43.997750 16565 tablet.cc:2369] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:43:44.073964 16833 tablet.cc:2369] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:44.074388 16855 raft_consensus.cc:482] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:44.074527 16855 raft_consensus.cc:504] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:44.074908 16855 leader_election.cc:288] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa0a9e94cd049f2acb13e2d4995273d (127.15.110.193:46501), 263484b697c443b38105ecde6088a956 (127.15.110.195:35597)
I20260927 18:43:44.078277 16845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.075346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:33371 (local address 127.15.110.195:35597)
0927 18:43:44.075399 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:44.075402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:44.075413 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:44.075448 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:44.075449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:44.075460 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:44.075516 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:44.075703 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.076242 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.077260 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.077845 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.077920 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.077929 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:44.078174 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 18:43:44.078209 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:43:44.078277 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.075270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54509 (local address 127.15.110.193:46501)
0927 18:43:44.075356 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:44.075365 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:44.075459 (+ 94us) server_negotiation.cc:407] Connection header received
0927 18:43:44.075527 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:44.075528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:44.075538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:44.075617 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:44.075797 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.076374 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.077417 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.077970 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.078040 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.078045 (+ 5us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:44.078170 (+ 125us) server_negotiation.cc:299] Negotiation successful
0927 18:43:44.078208 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260927 18:43:44.078294 16700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.075143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35597 (local address 127.15.110.194:33371)
0927 18:43:44.075225 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:44.075236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:44.075284 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:44.075533 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:44.075534 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:44.075536 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:44.075623 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:44.075625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.076309 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.076310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:44.077150 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:44.077153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.077978 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.077979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:44.078077 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.078080 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:44.078152 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:43:44.078193 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62,"mutex_wait_us":1}
I20260927 18:43:44.078351 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.075143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46501 (local address 127.15.110.194:54509)
0927 18:43:44.075385 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:44.075395 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:44.075447 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:44.075633 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:44.075633 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:44.075635 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:44.075721 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:44.075722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.076456 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.076457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:44.077339 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:44.077341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.078043 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.078044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:44.078101 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.078105 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:44.078160 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:43:44.078193 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"mutex_wait_us":23,"thread_start_us":130,"threads_started":1}
I20260927 18:43:44.078656 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa58a3d651b2468cbd12a383ec2b6821" candidate_uuid: "7c417bc0d226442ab7ed017fe0b41262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263484b697c443b38105ecde6088a956" is_pre_election: true
I20260927 18:43:44.078698 16518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa58a3d651b2468cbd12a383ec2b6821" candidate_uuid: "7c417bc0d226442ab7ed017fe0b41262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0a9e94cd049f2acb13e2d4995273d" is_pre_election: true
I20260927 18:43:44.078825 16786 raft_consensus.cc:2454] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c417bc0d226442ab7ed017fe0b41262 in term 0.
I20260927 18:43:44.078845 16518 raft_consensus.cc:2454] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c417bc0d226442ab7ed017fe0b41262 in term 0.
I20260927 18:43:44.079090 16588 leader_election.cc:302] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c417bc0d226442ab7ed017fe0b41262, baa0a9e94cd049f2acb13e2d4995273d; no voters:
I20260927 18:43:44.079263 16855 raft_consensus.cc:2781] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:44.079334 16855 raft_consensus.cc:482] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:44.079352 16855 raft_consensus.cc:3037] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:44.080052 16855 raft_consensus.cc:504] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:44.080190 16855 leader_election.cc:288] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [CANDIDATE]: Term 1 election: Requested vote from peers baa0a9e94cd049f2acb13e2d4995273d (127.15.110.193:46501), 263484b697c443b38105ecde6088a956 (127.15.110.195:35597)
I20260927 18:43:44.080366 16518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa58a3d651b2468cbd12a383ec2b6821" candidate_uuid: "7c417bc0d226442ab7ed017fe0b41262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa0a9e94cd049f2acb13e2d4995273d"
I20260927 18:43:44.080366 16786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aa58a3d651b2468cbd12a383ec2b6821" candidate_uuid: "7c417bc0d226442ab7ed017fe0b41262" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "263484b697c443b38105ecde6088a956"
I20260927 18:43:44.080440 16786 raft_consensus.cc:3037] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:44.080440 16518 raft_consensus.cc:3037] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:44.081277 16518 raft_consensus.cc:2454] T aa58a3d651b2468cbd12a383ec2b6821 P baa0a9e94cd049f2acb13e2d4995273d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c417bc0d226442ab7ed017fe0b41262 in term 1.
I20260927 18:43:44.081277 16786 raft_consensus.cc:2454] T aa58a3d651b2468cbd12a383ec2b6821 P 263484b697c443b38105ecde6088a956 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c417bc0d226442ab7ed017fe0b41262 in term 1.
I20260927 18:43:44.081476 16588 leader_election.cc:302] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c417bc0d226442ab7ed017fe0b41262, baa0a9e94cd049f2acb13e2d4995273d; no voters:
I20260927 18:43:44.081581 16855 raft_consensus.cc:2781] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:44.081732 16855 raft_consensus.cc:686] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [term 1 LEADER]: Becoming Leader. State: Replica: 7c417bc0d226442ab7ed017fe0b41262, State: Running, Role: LEADER
I20260927 18:43:44.081835 16855 consensus_queue.cc:237] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } }
I20260927 18:43:44.082767 16373 catalog_manager.cc:5495] T aa58a3d651b2468cbd12a383ec2b6821 P 7c417bc0d226442ab7ed017fe0b41262 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c417bc0d226442ab7ed017fe0b41262 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "7c417bc0d226442ab7ed017fe0b41262" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "baa0a9e94cd049f2acb13e2d4995273d" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46501 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "263484b697c443b38105ecde6088a956" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c417bc0d226442ab7ed017fe0b41262" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43557 } health_report { overall_health: HEALTHY } } }
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534623, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:44.095129 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.087660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51332 (local address 127.15.110.193:46501)
0927 18:43:44.087709 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:44.087712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:44.087780 (+ 68us) server_negotiation.cc:407] Connection header received
0927 18:43:44.087818 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:44.087819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:44.087828 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:44.087881 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:44.088391 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.088858 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.089562 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.089753 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.089815 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.092108 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:44.092118 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:44.092118 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:44.092125 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:44.093891 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.094241 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.094244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.094244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.094298 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.094562 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.094563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.094564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.094802 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:44.094936 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:44.094989 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 18:43:44.095064 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534624, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:44.117482 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.110629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51338 (local address 127.15.110.193:46501)
0927 18:43:44.110677 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:44.110679 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:44.110750 (+ 71us) server_negotiation.cc:407] Connection header received
0927 18:43:44.110795 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:44.110796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:44.110805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:44.110853 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:44.111448 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.111907 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.112659 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.112863 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.112931 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.115303 (+ 2372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:44.115305 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:44.115305 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:44.115311 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:44.116504 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.116810 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.116810 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.116811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.116858 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.117033 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.117034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.117034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.117121 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:44.117192 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:44.117331 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:43:44.117388 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534624, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:44.139462 16847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:44.132797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51350 (local address 127.15.110.193:46501)
0927 18:43:44.132869 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:44.132871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:44.132953 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:43:44.133048 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:44.133049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:44.133058 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:44.133120 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:44.133638 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.134097 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.134783 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:44.134954 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:44.135002 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:44.137234 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:44.137236 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:44.137236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:44.137242 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:44.138570 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.138851 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.138852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.138852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.138898 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:44.139040 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:44.139040 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:44.139040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:44.139129 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:44.139266 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:44.139333 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 18:43:44.139397 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 18:43:44.139947 16478 tablet_service.cc:2908] Rejecting scan request for tablet aa58a3d651b2468cbd12a383ec2b6821: Uninitialized: clean time has not yet been initialized
I20260927 18:43:44.140965 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16435
I20260927 18:43:44.147053 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16569
I20260927 18:43:44.151453 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16703
I20260927 18:43:44.155707 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16341
2026-09-27T18:43:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2148 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16873](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16873](info): set up 2 sockets
Sep 27 18:43:44 dist-test-slave-vs05 krb5kdc[16873](info): commencing operation
krb5kdc: starting...
W20260927 18:43:45.186683 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:45.309948 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46083
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:45.391633 16886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:45.391837 16886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:45.391860 16886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:45.393043 16886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:45.393088 16886 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:43:45.393101 16886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:45.393117 16886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:45.393131 16886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:45.393142 16886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:45.394723 16886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46083
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:45.394981 16886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:45.395264 16886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:45.400285 16892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:45.400314 16886 server_base.cc:1034] running on GCE node
W20260927 18:43:45.400285 16891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.400321 16894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:45.400712 16886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:45.400969 16886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:45.402140 16886 hybrid_clock.cc:648] HybridClock initialized: now 1790534625402127 us; error 32 us; skew 500 ppm
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:45.417910 16886 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:45.418395 16886 webserver.cc:460] Webserver started at http://127.15.110.254:45181/ using document root <none> and password file <none>
I20260927 18:43:45.418654 16886 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:45.418723 16886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:45.418872 16886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:45.420038 16886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "acdb310264a24683b8d9361ef2231a16"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "bbf1567f81337ab3a6e2364f11e371c0"
server_key_iv: "8280302a1a5d6540e6f8be162f092c0e"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.420408 16886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "acdb310264a24683b8d9361ef2231a16"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "bbf1567f81337ab3a6e2364f11e371c0"
server_key_iv: "8280302a1a5d6540e6f8be162f092c0e"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.421801 16886 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:45.422549 16901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.422755 16886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:45.422859 16886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "acdb310264a24683b8d9361ef2231a16"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "bbf1567f81337ab3a6e2364f11e371c0"
server_key_iv: "8280302a1a5d6540e6f8be162f092c0e"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.422945 16886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:45.441156 16886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:45.443336 16886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:45.443502 16886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:45.448148 16886 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46083
I20260927 18:43:45.448200 16963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46083 every 8 connection(s)
I20260927 18:43:45.448519 16886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:45.449157 16964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:45.451464 16964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Bootstrap starting.
I20260927 18:43:45.452066 16964 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:45.452356 16964 log.cc:824] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:45.453059 16964 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: No bootstrap required, opened a new log
I20260927 18:43:45.455096 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16886
I20260927 18:43:45.455224 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:45.455171 16964 raft_consensus.cc:348] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } }
I20260927 18:43:45.455390 15803 external_mini_cluster.cc:1428] Setting key 91db7c55ab1950998cc81c653bc95bea
I20260927 18:43:45.455412 16964 raft_consensus.cc:374] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:45.455466 16964 raft_consensus.cc:729] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acdb310264a24683b8d9361ef2231a16, State: Initialized, Role: FOLLOWER
I20260927 18:43:45.455624 16964 consensus_queue.cc:260] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } }
I20260927 18:43:45.455722 16964 raft_consensus.cc:388] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:45.455760 16964 raft_consensus.cc:482] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:45.455806 16964 raft_consensus.cc:3037] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:45.456477 16964 raft_consensus.cc:504] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } }
I20260927 18:43:45.456598 16964 leader_election.cc:302] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: acdb310264a24683b8d9361ef2231a16; no voters:
I20260927 18:43:45.456779 16964 leader_election.cc:288] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:45.456857 16969 raft_consensus.cc:2781] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:45.457052 16969 raft_consensus.cc:686] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [term 1 LEADER]: Becoming Leader. State: Replica: acdb310264a24683b8d9361ef2231a16, State: Running, Role: LEADER
I20260927 18:43:45.457053 16964 sys_catalog.cc:564] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:45.457190 16969 consensus_queue.cc:237] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } }
I20260927 18:43:45.457530 16969 sys_catalog.cc:455] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader acdb310264a24683b8d9361ef2231a16. Latest consensus state: current_term: 1 leader_uuid: "acdb310264a24683b8d9361ef2231a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } } }
I20260927 18:43:45.457763 16969 sys_catalog.cc:458] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:45.457690 16970 sys_catalog.cc:455] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "acdb310264a24683b8d9361ef2231a16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "acdb310264a24683b8d9361ef2231a16" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46083 } } }
I20260927 18:43:45.458182 16970 sys_catalog.cc:458] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:45.458290 16978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:45.458997 16978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:45.460633 16978 catalog_manager.cc:1347] Generated new cluster ID: f06dcf834b91493488c6b8d0339cccb3
I20260927 18:43:45.460685 16978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:45.466572 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.455863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.15.110.254:46083)
0927 18:43:45.456049 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.456058 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.456108 (+ 50us) server_negotiation.cc:407] Connection header received
0927 18:43:45.456200 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.456205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.456248 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.456360 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.457225 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.458788 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.459702 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.460288 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.460424 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.462759 (+ 2335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.462763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.462763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.462783 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.464897 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.465299 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.465302 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.465303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.465412 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.465651 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.465652 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.465653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.465902 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.466073 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.466177 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.466424 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":109,"threads_started":1}
I20260927 18:43:45.474880 16978 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:45.475538 16978 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:45.481160 16978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Generated new TSK 0
I20260927 18:43:45.481359 16978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:45.514405 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46083
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:45.597559 16990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:45.597760 16990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:45.597786 16990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:45.598891 16990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:45.598925 16990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:45.598951 16990 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:45.598969 16990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:45.600358 16990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:45.600551 16990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:45.600725 16990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:45.605806 16995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.605831 16996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.605806 16998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:45.606108 16990 server_base.cc:1034] running on GCE node
I20260927 18:43:45.606338 16990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:45.606637 16990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:45.607760 16990 hybrid_clock.cc:648] HybridClock initialized: now 1790534625607743 us; error 26 us; skew 500 ppm
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:45.611039 16990 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:45.611541 16990 webserver.cc:460] Webserver started at http://127.15.110.193:34723/ using document root <none> and password file <none>
I20260927 18:43:45.611797 16990 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:45.611866 16990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:45.612017 16990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:45.613072 16990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "f24b0db251eb4fedb9cd1e44f775daba"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f34577e3ed2cbadcee86ec162349e933"
server_key_iv: "9263a712ec5e83efcbaa99ab93f177f9"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.613444 16990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "f24b0db251eb4fedb9cd1e44f775daba"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f34577e3ed2cbadcee86ec162349e933"
server_key_iv: "9263a712ec5e83efcbaa99ab93f177f9"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.614836 16990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:45.615672 17005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.615850 16990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.615947 16990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "f24b0db251eb4fedb9cd1e44f775daba"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f34577e3ed2cbadcee86ec162349e933"
server_key_iv: "9263a712ec5e83efcbaa99ab93f177f9"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.616025 16990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:45.625890 16990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:45.628373 16990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:45.628615 16990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:45.629163 16990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:45.629232 16990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.629276 16990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:45.629307 16990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.635707 16990 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:43553
I20260927 18:43:45.635867 17120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:43553 every 8 connection(s)
I20260927 18:43:45.636126 16990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:45.639402 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.629596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:42795 (local address 127.15.110.254:46083)
0927 18:43:45.629642 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.629644 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.629847 (+ 203us) server_negotiation.cc:407] Connection header received
0927 18:43:45.630383 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.630384 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.630401 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.630453 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.631379 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.632231 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.633840 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.634181 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.634279 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.636908 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.636910 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.636910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.636917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.638142 (+ 1225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.638507 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.638507 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.638508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.638560 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.638743 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.638744 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.638744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.638846 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.638915 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.639151 (+ 236us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.639298 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:45.639405 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 16990
I20260927 18:43:45.639500 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:45.639496 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.629580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.193:42795)
0927 18:43:45.629732 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.629766 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.630269 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.630525 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.630530 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.630958 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.631259 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.631265 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.632339 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.632341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.633731 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.633735 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.634280 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.634281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.634390 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.635034 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.635045 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.636768 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.638215 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.638221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.638223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.638436 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.638602 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.638602 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.638602 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.638693 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.638921 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.638926 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.639037 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.639134 (+ 97us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.639316 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":82,"threads_started":1}
I20260927 18:43:45.639689 15803 external_mini_cluster.cc:1428] Setting key d96f5dc9c70690f6c4acc63c0963c319
I20260927 18:43:45.640093 17121 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46083
I20260927 18:43:45.640187 17121 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:45.640362 17121 heartbeater.cc:511] Master 127.15.110.254:46083 requested a full tablet report, sending...
I20260927 18:43:45.640967 16917 ts_manager.cc:194] Registered new tserver with Master: f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553)
I20260927 18:43:45.641788 16917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:42795
I20260927 18:43:45.648763 17127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.641182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.193:37455)
0927 18:43:45.641353 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.641365 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.641420 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.642170 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.642171 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.642421 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.642494 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.642495 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.643403 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.643404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.643916 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.643918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.644381 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.644382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.644489 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.644936 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.644946 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.645359 (+ 413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.647603 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.647604 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.647604 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.647798 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.648102 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.648103 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.648103 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.648175 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.648483 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.648484 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.648528 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.648590 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.648678 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260927 18:43:45.648838 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.642002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:37455 (local address 127.15.110.254:46083)
0927 18:43:45.642044 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.642046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.642053 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:43:45.642073 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.642073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.642082 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.642143 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.642603 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.643309 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.644030 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.644280 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.644353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.645536 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.645538 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.645539 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.645547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.647466 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.647924 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.647925 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.647926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.647994 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.648253 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.648254 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.648255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.648402 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.648477 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.648644 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.648732 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:45.670130 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46083
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:45.751093 17130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:45.751394 17130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:45.751433 17130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:45.752631 17130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:45.752704 17130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:45.752741 17130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:45.752779 17130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:45.754226 17130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:45.754522 17130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:45.754755 17130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:45.759864 17138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.759933 17135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.759896 17136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:45.760213 17130 server_base.cc:1034] running on GCE node
I20260927 18:43:45.760442 17130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:45.760679 17130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:45.761823 17130 hybrid_clock.cc:648] HybridClock initialized: now 1790534625761803 us; error 29 us; skew 500 ppm
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:45.764249 17130 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:45.764783 17130 webserver.cc:460] Webserver started at http://127.15.110.194:46645/ using document root <none> and password file <none>
I20260927 18:43:45.765051 17130 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:45.765136 17130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:45.765283 17130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:45.766332 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "a604b5d8b46249128f5701d4468faac8"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f436466a80d541dd5c0f66be84207bb0"
server_key_iv: "503aa15e5ca87aa4293d675890bc457d"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.766688 17130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "a604b5d8b46249128f5701d4468faac8"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f436466a80d541dd5c0f66be84207bb0"
server_key_iv: "503aa15e5ca87aa4293d675890bc457d"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.768107 17130 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:45.768884 17145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.769080 17130 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:45.769193 17130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "a604b5d8b46249128f5701d4468faac8"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "f436466a80d541dd5c0f66be84207bb0"
server_key_iv: "503aa15e5ca87aa4293d675890bc457d"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.769268 17130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:45.779898 17130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:45.781745 17130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:45.781916 17130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:45.782430 17130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:45.782500 17130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.782544 17130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:45.782575 17130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.789448 17130 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:45233
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:45.789508 17260 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:45233 every 8 connection(s)
I20260927 18:43:45.789811 17130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:45.792462 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.783049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:36189 (local address 127.15.110.254:46083)
0927 18:43:45.783097 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.783100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.783342 (+ 242us) server_negotiation.cc:407] Connection header received
0927 18:43:45.783975 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.783976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.783989 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.784042 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.785047 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.785898 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.786949 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.787315 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.787396 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.789989 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.789990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.789991 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.789999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.791355 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.791749 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.791750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.791750 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.791802 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.791969 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.791970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.791970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.792070 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.792120 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.792341 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.792401 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:43:45.792726 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.782995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.194:36189)
0927 18:43:45.783200 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.783245 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.783873 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.784181 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.784186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.784592 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.784931 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.784938 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.786043 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.786044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.786844 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.786846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.787414 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.787414 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.787624 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.788185 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.788194 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.789875 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.791423 (+ 1548us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.791429 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.791430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.791682 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.791843 (+ 161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.791844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.791844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.791921 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.792126 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.792130 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.792234 (+ 104us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.792314 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.792424 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":92,"threads_started":1}
I20260927 18:43:45.793218 17261 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46083
I20260927 18:43:45.793301 17261 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:45.793479 17261 heartbeater.cc:511] Master 127.15.110.254:46083 requested a full tablet report, sending...
I20260927 18:43:45.793841 16917 ts_manager.cc:194] Registered new tserver with Master: a604b5d8b46249128f5701d4468faac8 (127.15.110.194:45233)
I20260927 18:43:45.794333 16917 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:36189
I20260927 18:43:45.795162 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17130
I20260927 18:43:45.795271 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:45.795446 15803 external_mini_cluster.cc:1428] Setting key de1c6c40aaff6bf776254c94ae0a519a
I20260927 18:43:45.800422 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.794293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:36121 (local address 127.15.110.254:46083)
0927 18:43:45.794344 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.794346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.794413 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:43:45.794483 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.794484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.794494 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.794656 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.795006 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.795649 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.796534 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.796852 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.796934 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.797829 (+ 895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.797830 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.797830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.797836 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.799147 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.799554 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.799555 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.799556 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.799636 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.799844 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.799845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.799845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.799998 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.800075 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.800259 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.800327 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:45.800426 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.794226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.194:36121)
0927 18:43:45.794343 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.794355 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.794434 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.794665 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.794666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.794851 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.794900 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.794901 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.795736 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.795737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.796446 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.796448 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.796931 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.796932 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.796991 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.797437 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.797442 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.797719 (+ 277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.799291 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.799292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.799293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.799472 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.799710 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.799711 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.799712 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.799778 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.800077 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.800078 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.800144 (+ 66us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.800222 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.800328 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:45.823323 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46083
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:35115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:45.905069 17270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:45.905292 17270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:45.905315 17270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:45.906433 17270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:45.906473 17270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:45.906493 17270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:45.906513 17270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:45.908004 17270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46083
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:45.908192 17270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:45.908377 17270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:43:45.914006 17270 server_base.cc:1034] running on GCE node
W20260927 18:43:45.913977 17276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.913987 17275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:45.914014 17278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:45.914391 17270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:45.914601 17270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:45.915745 17270 hybrid_clock.cc:648] HybridClock initialized: now 1790534625915723 us; error 36 us; skew 500 ppm
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:45.918215 17270 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:45.918697 17270 webserver.cc:460] Webserver started at http://127.15.110.195:35903/ using document root <none> and password file <none>
I20260927 18:43:45.918936 17270 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:45.919001 17270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:45.919157 17270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:45.920248 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "92f0f405454541b381765455a60d913c"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "d022dd6e0191a3eb40f99c0f1c636b21"
server_key_iv: "1ce2578cc54180073255a42b35bb9a02"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.920639 17270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "92f0f405454541b381765455a60d913c"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "d022dd6e0191a3eb40f99c0f1c636b21"
server_key_iv: "1ce2578cc54180073255a42b35bb9a02"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.921970 17270 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:45.922734 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.922930 17270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:45.923035 17270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "92f0f405454541b381765455a60d913c"
format_stamp: "Formatted at 2026-09-27 18:43:45 on dist-test-slave-vs05"
server_key: "d022dd6e0191a3eb40f99c0f1c636b21"
server_key_iv: "1ce2578cc54180073255a42b35bb9a02"
server_key_version: "encryptionkey@0"
I20260927 18:43:45.923125 17270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:45.940450 17270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:45.942487 17270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:45.942659 17270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:45.943269 17270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:45.943329 17270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:45.943369 17270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:45.943397 17270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:45.950736 17270 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:35539
I20260927 18:43:45.950798 17400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:35539 every 8 connection(s)
I20260927 18:43:45.951153 17270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:45.954186 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.943786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:59877 (local address 127.15.110.254:46083)
0927 18:43:45.943861 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.943863 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.944134 (+ 271us) server_negotiation.cc:407] Connection header received
0927 18:43:45.944904 (+ 770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.944905 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.944915 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.944956 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.945932 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.946703 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.947939 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.948274 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.948351 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.951200 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.951202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.951202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.951210 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.952724 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.953167 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.953168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.953168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.953237 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.953489 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.953490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.953491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.953618 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.953696 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.954004 (+ 308us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.954098 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:43:45.954375 17294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.943760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.195:59877)
0927 18:43:45.943980 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.944032 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.944770 (+ 738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.945064 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.945069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.945508 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.945846 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.945854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.946800 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.946801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.947831 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.947835 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.948369 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.948371 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.948565 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.949123 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.949136 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.951071 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.952816 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.952822 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.952825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.953089 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.953305 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.953306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.953306 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.953424 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.953687 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.953693 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.953861 (+ 168us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.953974 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.954143 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260927 18:43:45.954879 17401 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46083
I20260927 18:43:45.954968 17401 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:45.955232 17401 heartbeater.cc:511] Master 127.15.110.254:46083 requested a full tablet report, sending...
I20260927 18:43:45.955631 16918 ts_manager.cc:194] Registered new tserver with Master: 92f0f405454541b381765455a60d913c (127.15.110.195:35539)
I20260927 18:43:45.956059 16918 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:59877
I20260927 18:43:45.958850 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17270
I20260927 18:43:45.958964 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:45.959213 15803 external_mini_cluster.cc:1428] Setting key fa08f7442bbb89c16ad3b6253649410b
I20260927 18:43:45.960081 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:45.961743 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.956037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49417 (local address 127.15.110.254:46083)
0927 18:43:45.956081 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.956083 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.956167 (+ 84us) server_negotiation.cc:407] Connection header received
0927 18:43:45.956273 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.956274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.956284 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.956345 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.956749 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.957227 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.957990 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.958175 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.958228 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.959224 (+ 996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.959225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.959226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.959234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.960638 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.960985 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.960986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.960987 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.961046 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.961238 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.961239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.961239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.961352 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.961423 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.961578 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.961654 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:45.961758 17407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.955983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.15.110.195:49417)
0927 18:43:45.956101 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.956114 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.956195 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.956373 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.956374 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.956569 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:45.956631 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.956633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.957308 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.957308 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.957926 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.957927 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.958250 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.958251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.958331 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.958790 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:45.958794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:45.959115 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:45.960735 (+ 1620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.960736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.960736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.960902 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.961113 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:45.961113 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:45.961114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:45.961172 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:45.961428 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:45.961429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:45.961472 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.961552 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.961655 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":64,"threads_started":1}
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:43:45 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:45.983160 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.976205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60864 (local address 127.15.110.254:46083)
0927 18:43:45.976291 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.976293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.976299 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:43:45.976321 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.976322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.976330 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.976398 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:45.976958 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.977452 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.978171 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.978370 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.978448 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.980795 (+ 2347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:45.980797 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:45.980798 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:45.980804 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:45.982168 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.982490 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.982491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.982492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.982541 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:45.982705 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:45.982706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:45.982706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:45.982803 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:45.982874 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.982978 (+ 104us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.983037 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:43:45.984733 16918 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:45.986285 16918 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:45.992408 17420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43553 (local address 127.0.0.1:56578)
0927 18:43:45.988878 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.988926 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.989000 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.989489 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.989493 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.989497 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:45.989594 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.989597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.990471 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.990471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.991394 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.991397 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.992093 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.992094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.992173 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.992188 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.992263 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.992318 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":26,"thread_start_us":55,"threads_started":1}
I20260927 18:43:45.992427 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56578 (local address 127.15.110.193:43553)
0927 18:43:45.988884 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.988890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.989264 (+ 374us) server_negotiation.cc:407] Connection header received
0927 18:43:45.989311 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.989313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.989329 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.989405 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:45.989687 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.990400 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.991481 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.991986 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.992093 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.992173 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.992283 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.992321 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260927 18:43:45.993439 17055 tablet_service.cc:1467] Processing CreateTablet for tablet 4d387d9249a44b4fbb63b5caedababea (DEFAULT_TABLE table=test-table [id=68f90819d90944a7a3901e770128de8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:45.993676 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40532 (local address 127.15.110.195:35539)
0927 18:43:45.989032 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.989037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.989353 (+ 316us) server_negotiation.cc:407] Connection header received
0927 18:43:45.989398 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.989401 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.989417 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.989493 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:45.990487 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.991243 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.992547 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.993273 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.993346 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.993390 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.993526 (+ 136us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.993580 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":88,"threads_started":1}
I20260927 18:43:45.993886 17055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d387d9249a44b4fbb63b5caedababea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:45.993909 17419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35539 (local address 127.0.0.1:40532)
0927 18:43:45.989147 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.989156 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.989201 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.989564 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.989565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.989566 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:45.989608 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.989610 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.991349 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.991350 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.992468 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.992469 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.993356 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.993357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.993419 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.993425 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.993781 (+ 356us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.993834 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":80,"threads_started":1}
I20260927 18:43:45.994642 17418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45233 (local address 127.0.0.1:55550)
0927 18:43:45.989124 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:45.989132 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:45.989224 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:45.989490 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:45.989493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:45.989495 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:45.989594 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.989596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.992443 (+ 2847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.992444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.993514 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:45.993517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.994389 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.994390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:45.994463 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.994467 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:45.994525 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:43:45.994561 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
I20260927 18:43:45.994656 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 4d387d9249a44b4fbb63b5caedababea (DEFAULT_TABLE table=test-table [id=68f90819d90944a7a3901e770128de8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:45.994872 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:45.988937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.15.110.194:45233)
0927 18:43:45.989135 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:45.989141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:45.989211 (+ 70us) server_negotiation.cc:407] Connection header received
0927 18:43:45.989311 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:45.989313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:45.989329 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:45.989405 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:45.991226 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.992350 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.993602 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:45.994305 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:45.994378 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:45.994423 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:45.994741 (+ 318us) server_negotiation.cc:299] Negotiation successful
0927 18:43:45.994785 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":90,"threads_started":1}
I20260927 18:43:45.994992 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d387d9249a44b4fbb63b5caedababea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:45.995662 17195 tablet_service.cc:1467] Processing CreateTablet for tablet 4d387d9249a44b4fbb63b5caedababea (DEFAULT_TABLE table=test-table [id=68f90819d90944a7a3901e770128de8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:45.995990 17195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4d387d9249a44b4fbb63b5caedababea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:45.997797 17424 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Bootstrap starting.
I20260927 18:43:45.998142 17425 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Bootstrap starting.
I20260927 18:43:45.998839 17424 tablet_bootstrap.cc:654] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:45.998839 17425 tablet_bootstrap.cc:654] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:45.999241 17424 log.cc:824] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:45.999241 17425 log.cc:824] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:46.000052 17425 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: No bootstrap required, opened a new log
I20260927 18:43:46.000034 17428 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Bootstrap starting.
I20260927 18:43:46.000124 17425 ts_tablet_manager.cc:1397] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:43:46.000309 17424 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: No bootstrap required, opened a new log
I20260927 18:43:46.000391 17424 ts_tablet_manager.cc:1397] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260927 18:43:46.000681 17428 tablet_bootstrap.cc:654] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.000998 17428 log.cc:824] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:46.001792 17428 tablet_bootstrap.cc:492] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: No bootstrap required, opened a new log
I20260927 18:43:46.001861 17428 ts_tablet_manager.cc:1397] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:46.002192 17424 raft_consensus.cc:348] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.002363 17424 raft_consensus.cc:374] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.002254 17425 raft_consensus.cc:348] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.002420 17424 raft_consensus.cc:729] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f0f405454541b381765455a60d913c, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.002424 17425 raft_consensus.cc:374] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.002461 17425 raft_consensus.cc:729] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24b0db251eb4fedb9cd1e44f775daba, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.002566 17424 consensus_queue.cc:260] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.002575 17425 consensus_queue.cc:260] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.002856 17424 ts_tablet_manager.cc:1428] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:43:46.002852 17425 ts_tablet_manager.cc:1428] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:43:46.002997 17401 heartbeater.cc:503] Master 127.15.110.254:46083 was elected leader, sending a full tablet report...
I20260927 18:43:46.002993 17121 heartbeater.cc:503] Master 127.15.110.254:46083 was elected leader, sending a full tablet report...
I20260927 18:43:46.004026 17428 raft_consensus.cc:348] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.004202 17428 raft_consensus.cc:374] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.004245 17428 raft_consensus.cc:729] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a604b5d8b46249128f5701d4468faac8, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.004364 17428 consensus_queue.cc:260] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.004621 17428 ts_tablet_manager.cc:1428] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:46.004673 17261 heartbeater.cc:503] Master 127.15.110.254:46083 was elected leader, sending a full tablet report...
W20260927 18:43:46.040390 17262 tablet.cc:2369] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:46.135327 17431 raft_consensus.cc:482] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:46.135553 17431 raft_consensus.cc:504] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.135980 17431 leader_election.cc:288] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92f0f405454541b381765455a60d913c (127.15.110.195:35539), a604b5d8b46249128f5701d4468faac8 (127.15.110.194:45233)
W20260927 18:43:46.136943 17122 tablet.cc:2369] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:46.140079 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.136299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:40435 (local address 127.15.110.194:45233)
0927 18:43:46.136353 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.136356 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.136459 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:43:46.136487 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.136488 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.136497 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.136551 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:46.136768 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.137501 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139105 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.139687 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139762 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.139772 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.139980 (+ 208us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.140016 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:43:46.140079 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.136294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:51545 (local address 127.15.110.195:35539)
0927 18:43:46.136341 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.136344 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.136356 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:43:46.136386 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.136387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.136397 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.136471 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:46.136690 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.137558 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139103 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.139732 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139797 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.139804 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.139962 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.140005 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:46.140110 17014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.136193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35539 (local address 127.15.110.193:51545)
0927 18:43:46.136259 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.136272 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.136316 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.136491 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.136492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.136494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:46.136588 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.136589 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.137654 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.137655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.138997 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.139000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.139797 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.139877 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.139881 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.139943 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.139987 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":82}
I20260927 18:43:46.140336 17433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.136193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45233 (local address 127.15.110.193:40435)
0927 18:43:46.136343 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.136352 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.136410 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.136570 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.136570 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.136571 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:46.136660 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.136662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.137654 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.137655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.138997 (+ 1342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.139000 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.139789 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.139790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.139877 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.139882 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.139943 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.139992 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":92,"thread_start_us":77,"threads_started":1}
I20260927 18:43:46.140518 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d387d9249a44b4fbb63b5caedababea" candidate_uuid: "f24b0db251eb4fedb9cd1e44f775daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c" is_pre_election: true
I20260927 18:43:46.140697 17355 raft_consensus.cc:2454] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f24b0db251eb4fedb9cd1e44f775daba in term 0.
I20260927 18:43:46.140735 17215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d387d9249a44b4fbb63b5caedababea" candidate_uuid: "f24b0db251eb4fedb9cd1e44f775daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a604b5d8b46249128f5701d4468faac8" is_pre_election: true
I20260927 18:43:46.140903 17215 raft_consensus.cc:2454] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f24b0db251eb4fedb9cd1e44f775daba in term 0.
I20260927 18:43:46.140961 17006 leader_election.cc:302] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f0f405454541b381765455a60d913c, f24b0db251eb4fedb9cd1e44f775daba; no voters:
I20260927 18:43:46.141089 17431 raft_consensus.cc:2781] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:46.141165 17431 raft_consensus.cc:482] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:46.141183 17431 raft_consensus.cc:3037] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.142119 17431 raft_consensus.cc:504] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.142264 17431 leader_election.cc:288] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [CANDIDATE]: Term 1 election: Requested vote from peers 92f0f405454541b381765455a60d913c (127.15.110.195:35539), a604b5d8b46249128f5701d4468faac8 (127.15.110.194:45233)
I20260927 18:43:46.142431 17215 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d387d9249a44b4fbb63b5caedababea" candidate_uuid: "f24b0db251eb4fedb9cd1e44f775daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a604b5d8b46249128f5701d4468faac8"
I20260927 18:43:46.142498 17215 raft_consensus.cc:3037] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.143251 17215 raft_consensus.cc:2454] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f24b0db251eb4fedb9cd1e44f775daba in term 1.
I20260927 18:43:46.143271 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4d387d9249a44b4fbb63b5caedababea" candidate_uuid: "f24b0db251eb4fedb9cd1e44f775daba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c"
I20260927 18:43:46.143371 17355 raft_consensus.cc:3037] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.143436 17008 leader_election.cc:302] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a604b5d8b46249128f5701d4468faac8, f24b0db251eb4fedb9cd1e44f775daba; no voters:
I20260927 18:43:46.143522 17431 raft_consensus.cc:2781] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:46.143662 17431 raft_consensus.cc:686] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 1 LEADER]: Becoming Leader. State: Replica: f24b0db251eb4fedb9cd1e44f775daba, State: Running, Role: LEADER
I20260927 18:43:46.143877 17431 consensus_queue.cc:237] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } }
I20260927 18:43:46.144280 17355 raft_consensus.cc:2454] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f24b0db251eb4fedb9cd1e44f775daba in term 1.
I20260927 18:43:46.144802 16916 catalog_manager.cc:5495] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba reported cstate change: term changed from 0 to 1, leader changed from <none> to f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193). New cstate: current_term: 1 leader_uuid: "f24b0db251eb4fedb9cd1e44f775daba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:46.175717 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.172940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.15.110.193:43553)
0927 18:43:46.173006 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.173008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.173024 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:43:46.173066 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.173067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.173085 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.173150 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:46.173295 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.173823 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.174728 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.174972 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.175046 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.175343 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.175445 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.175509 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.175560 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.175598 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:43:46.177523 17215 raft_consensus.cc:1261] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Refusing update from remote peer f24b0db251eb4fedb9cd1e44f775daba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.177563 17355 raft_consensus.cc:1261] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Refusing update from remote peer f24b0db251eb4fedb9cd1e44f775daba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.177902 17431 consensus_queue.cc:1035] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [LEADER]: Connected to new peer: Peer: permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.178053 17434 consensus_queue.cc:1035] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.180462 17437 mvcc.cc:205] Tried to move back new op lower bound from 7334029828820586496 to 7334029828686036992. Current Snapshot: MvccSnapshot[applied={T|T < 7334029828820586496}]
I20260927 18:43:46.180708 17438 mvcc.cc:205] Tried to move back new op lower bound from 7334029828820586496 to 7334029828686036992. Current Snapshot: MvccSnapshot[applied={T|T < 7334029828820586496}]
I20260927 18:43:46.184079 16916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:46.184182 16916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:46.184986 16916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Sending DeleteTablet for 3 replicas of tablet 4d387d9249a44b4fbb63b5caedababea
I20260927 18:43:46.185345 17055 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d387d9249a44b4fbb63b5caedababea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:56578
I20260927 18:43:46.185567 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d387d9249a44b4fbb63b5caedababea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:40532
I20260927 18:43:46.185631 16916 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:46.185768 16916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:46.185782 17448 tablet_replica.cc:330] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: stopping tablet replica
I20260927 18:43:46.185884 17448 raft_consensus.cc:2229] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:46.185892 17195 tablet_service.cc:1514] Processing DeleteTablet for tablet 4d387d9249a44b4fbb63b5caedababea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55550
I20260927 18:43:46.186000 17448 raft_consensus.cc:2258] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.186187 17447 tablet_replica.cc:330] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: stopping tablet replica
I20260927 18:43:46.186272 17447 raft_consensus.cc:2229] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:46.186384 17447 raft_consensus.cc:2258] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.186429 17448 ts_tablet_manager.cc:1905] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.186787 17447 ts_tablet_manager.cc:1905] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.187698 17195 tablet_service.cc:1467] Processing CreateTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 (DEFAULT_TABLE table=test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:46.187870 17449 tablet_replica.cc:330] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: stopping tablet replica
I20260927 18:43:46.187944 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 (DEFAULT_TABLE table=test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:46.188042 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ba2471503f4279a8e0cb0a9775b966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.188127 17449 raft_consensus.cc:2229] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:46.188280 17449 raft_consensus.cc:2258] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.188665 17448 ts_tablet_manager.cc:1918] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:46.188740 17448 log.cc:1196] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/4d387d9249a44b4fbb63b5caedababea
I20260927 18:43:46.188905 17449 ts_tablet_manager.cc:1905] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.189007 17448 ts_tablet_manager.cc:1939] T 4d387d9249a44b4fbb63b5caedababea P 92f0f405454541b381765455a60d913c: Deleting consensus metadata
I20260927 18:43:46.189292 17424 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Bootstrap starting.
I20260927 18:43:46.189369 17055 tablet_service.cc:1467] Processing CreateTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 (DEFAULT_TABLE table=test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:46.189463 17055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ba2471503f4279a8e0cb0a9775b966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.189710 17424 tablet_bootstrap.cc:654] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.190117 16902 catalog_manager.cc:4841] TS 92f0f405454541b381765455a60d913c (127.15.110.195:35539): tablet 4d387d9249a44b4fbb63b5caedababea (table test-table [id=68f90819d90944a7a3901e770128de8b]) successfully deleted
I20260927 18:43:46.190304 17424 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: No bootstrap required, opened a new log
I20260927 18:43:46.190351 17424 ts_tablet_manager.cc:1397] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:46.190503 17424 raft_consensus.cc:348] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.190584 17424 raft_consensus.cc:374] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.190624 17424 raft_consensus.cc:729] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f0f405454541b381765455a60d913c, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.190677 17425 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Bootstrap starting.
I20260927 18:43:46.190675 17424 consensus_queue.cc:260] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.190779 17424 ts_tablet_manager.cc:1428] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:46.191031 17425 tablet_bootstrap.cc:654] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.187875 17195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95ba2471503f4279a8e0cb0a9775b966. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.191661 17425 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: No bootstrap required, opened a new log
I20260927 18:43:46.191711 17425 ts_tablet_manager.cc:1397] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:46.191849 17425 raft_consensus.cc:348] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.191920 17425 raft_consensus.cc:374] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.191941 17425 raft_consensus.cc:729] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24b0db251eb4fedb9cd1e44f775daba, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.191987 17425 consensus_queue.cc:260] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.192066 17425 ts_tablet_manager.cc:1428] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:46.192679 17428 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Bootstrap starting.
I20260927 18:43:46.193032 17428 tablet_bootstrap.cc:654] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.193557 17428 tablet_bootstrap.cc:492] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: No bootstrap required, opened a new log
I20260927 18:43:46.193596 17428 ts_tablet_manager.cc:1397] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:46.193739 17428 raft_consensus.cc:348] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.193814 17428 raft_consensus.cc:374] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.193830 17428 raft_consensus.cc:729] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a604b5d8b46249128f5701d4468faac8, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.193884 17428 consensus_queue.cc:260] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.194000 17428 ts_tablet_manager.cc:1428] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:46.194970 17449 ts_tablet_manager.cc:1918] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:46.195092 17449 log.cc:1196] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/4d387d9249a44b4fbb63b5caedababea
I20260927 18:43:46.195293 17447 ts_tablet_manager.cc:1918] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:43:46.195358 17447 log.cc:1196] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/4d387d9249a44b4fbb63b5caedababea
I20260927 18:43:46.195511 17449 ts_tablet_manager.cc:1939] T 4d387d9249a44b4fbb63b5caedababea P a604b5d8b46249128f5701d4468faac8: Deleting consensus metadata
I20260927 18:43:46.195549 17447 ts_tablet_manager.cc:1939] T 4d387d9249a44b4fbb63b5caedababea P f24b0db251eb4fedb9cd1e44f775daba: Deleting consensus metadata
I20260927 18:43:46.195859 16903 catalog_manager.cc:4841] TS f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553): tablet 4d387d9249a44b4fbb63b5caedababea (table test-table [id=68f90819d90944a7a3901e770128de8b]) successfully deleted
I20260927 18:43:46.196103 16904 catalog_manager.cc:4841] TS a604b5d8b46249128f5701d4468faac8 (127.15.110.194:45233): tablet 4d387d9249a44b4fbb63b5caedababea (table test-table [id=68f90819d90944a7a3901e770128de8b]) successfully deleted
W20260927 18:43:46.201843 17402 tablet.cc:2369] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:46.248050 17432 raft_consensus.cc:482] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:46.248171 17432 raft_consensus.cc:504] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.248533 17432 leader_election.cc:288] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553), 92f0f405454541b381765455a60d913c (127.15.110.195:35539)
I20260927 18:43:46.252007 17154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.248717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35539 (local address 127.15.110.194:59173)
0927 18:43:46.248794 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.248807 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.248855 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.249033 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.249034 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.249036 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:46.249111 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.249113 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.249771 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.249772 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.250984 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.251002 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.251728 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.251729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.251806 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.251810 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.251870 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.251916 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 18:43:46.251993 17422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.248843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:59173 (local address 127.15.110.195:35539)
0927 18:43:46.248885 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.248887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.248895 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:46.248921 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.248922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.248930 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.248984 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:46.249177 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.249679 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.251112 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.251636 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.251722 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.251732 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.251889 (+ 157us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.251929 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:43:46.252244 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ba2471503f4279a8e0cb0a9775b966" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c" is_pre_election: true
I20260927 18:43:46.252339 17355 raft_consensus.cc:2454] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 0.
I20260927 18:43:46.252346 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.248874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:43863 (local address 127.15.110.193:43553)
0927 18:43:46.248920 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.248923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.248992 (+ 69us) server_negotiation.cc:407] Connection header received
0927 18:43:46.249070 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.249071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.249081 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.249148 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:46.249913 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.250361 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.251428 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.251945 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.252020 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.252028 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.252239 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.252279 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:43:46.252573 17453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.248725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43553 (local address 127.15.110.194:43863)
0927 18:43:46.248943 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.248952 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.249012 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.249758 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.249759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.249761 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:46.249829 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.249830 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.250449 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.250450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.251339 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.251342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.252064 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.252065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.252147 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.252152 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.252430 (+ 278us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.252475 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":14,"thread_start_us":65,"threads_started":1}
I20260927 18:43:46.252558 17146 leader_election.cc:302] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f0f405454541b381765455a60d913c, a604b5d8b46249128f5701d4468faac8; no voters:
I20260927 18:43:46.252724 17432 raft_consensus.cc:2781] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:46.252763 17432 raft_consensus.cc:482] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:46.252784 17432 raft_consensus.cc:3037] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.252916 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ba2471503f4279a8e0cb0a9775b966" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b0db251eb4fedb9cd1e44f775daba" is_pre_election: true
I20260927 18:43:46.253049 17075 raft_consensus.cc:2454] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 0.
I20260927 18:43:46.253685 17432 raft_consensus.cc:504] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.253831 17432 leader_election.cc:288] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 election: Requested vote from peers f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553), 92f0f405454541b381765455a60d913c (127.15.110.195:35539)
I20260927 18:43:46.254004 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ba2471503f4279a8e0cb0a9775b966" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c"
I20260927 18:43:46.254042 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95ba2471503f4279a8e0cb0a9775b966" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b0db251eb4fedb9cd1e44f775daba"
I20260927 18:43:46.254091 17355 raft_consensus.cc:3037] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.254115 17075 raft_consensus.cc:3037] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.254758 17075 raft_consensus.cc:2454] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 1.
I20260927 18:43:46.254844 17355 raft_consensus.cc:2454] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 1.
I20260927 18:43:46.254937 17147 leader_election.cc:302] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a604b5d8b46249128f5701d4468faac8, f24b0db251eb4fedb9cd1e44f775daba; no voters:
I20260927 18:43:46.255038 17432 raft_consensus.cc:2781] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:46.255262 17432 raft_consensus.cc:686] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 1 LEADER]: Becoming Leader. State: Replica: a604b5d8b46249128f5701d4468faac8, State: Running, Role: LEADER
I20260927 18:43:46.255343 17432 consensus_queue.cc:237] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.256120 16916 catalog_manager.cc:5495] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a604b5d8b46249128f5701d4468faac8 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:43:46 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:46.300027 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.292127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.15.110.254:46083)
0927 18:43:46.292182 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.292184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.292295 (+ 111us) server_negotiation.cc:407] Connection header received
0927 18:43:46.292363 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.292364 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.292372 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.292425 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:46.293049 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.293632 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.294662 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.294858 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.294915 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.297276 (+ 2361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:46.297278 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:46.297278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:46.297285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:46.298753 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.299213 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.299214 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.299215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.299276 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.299470 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.299471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.299471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.299585 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:46.299661 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.299816 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.299908 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:43:46.300103 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.292000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46083 (local address 127.0.0.1:60872)
0927 18:43:46.292207 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.292220 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.292301 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.292457 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.292458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.292852 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:46.292911 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.292914 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.293726 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.293728 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.294565 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.294567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.294941 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.294942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.295010 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.295598 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:46.295603 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:46.297181 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:46.298871 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:46.298881 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:46.298883 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:46.299090 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:46.299340 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:46.299341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:46.299341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:46.299407 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:46.299651 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:46.299659 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:46.299705 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.299794 (+ 89us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.299976 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":75,"thread_start_us":111,"threads_started":1}
I20260927 18:43:46.302704 16916 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:60872:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:43:46.302961 16916 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:46.304540 17195 tablet_service.cc:1467] Processing CreateTablet for tablet 002b1f040bdd43f1a90a75d57c7416bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d90da67ba5e4be6aacb33c9e26236de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:46.304538 17055 tablet_service.cc:1467] Processing CreateTablet for tablet 002b1f040bdd43f1a90a75d57c7416bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d90da67ba5e4be6aacb33c9e26236de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:46.304698 17055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b1f040bdd43f1a90a75d57c7416bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.304698 17195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b1f040bdd43f1a90a75d57c7416bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.306303 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 002b1f040bdd43f1a90a75d57c7416bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2d90da67ba5e4be6aacb33c9e26236de]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:46.306443 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 002b1f040bdd43f1a90a75d57c7416bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:46.307340 17425 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: Bootstrap starting.
I20260927 18:43:46.307509 17424 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: Bootstrap starting.
I20260927 18:43:46.307858 17425 tablet_bootstrap.cc:654] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.307925 17424 tablet_bootstrap.cc:654] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.308367 17428 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: Bootstrap starting.
I20260927 18:43:46.308657 17424 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: No bootstrap required, opened a new log
I20260927 18:43:46.308704 17424 ts_tablet_manager.cc:1397] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:46.308761 17428 tablet_bootstrap.cc:654] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:46.308815 17425 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: No bootstrap required, opened a new log
I20260927 18:43:46.308854 17425 ts_tablet_manager.cc:1397] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:46.308876 17424 raft_consensus.cc:348] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.308965 17424 raft_consensus.cc:374] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.308943 17425 raft_consensus.cc:348] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.308987 17424 raft_consensus.cc:729] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92f0f405454541b381765455a60d913c, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.309007 17425 raft_consensus.cc:374] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.309031 17425 raft_consensus.cc:729] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f24b0db251eb4fedb9cd1e44f775daba, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.309036 17424 consensus_queue.cc:260] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.309072 17425 consensus_queue.cc:260] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.309129 17424 ts_tablet_manager.cc:1428] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:46.309154 17425 ts_tablet_manager.cc:1428] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:46.309273 17431 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.309362 17431 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.309588 17428 tablet_bootstrap.cc:492] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: No bootstrap required, opened a new log
I20260927 18:43:46.309635 17428 ts_tablet_manager.cc:1397] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:46.309777 17428 raft_consensus.cc:348] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.309847 17428 raft_consensus.cc:374] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:46.309871 17428 raft_consensus.cc:729] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a604b5d8b46249128f5701d4468faac8, State: Initialized, Role: FOLLOWER
I20260927 18:43:46.309916 17428 consensus_queue.cc:260] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.310027 17428 ts_tablet_manager.cc:1428] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:46.310070 17432 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.310148 17432 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.309655 17430 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.310873 17430 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.390580 17432 raft_consensus.cc:482] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:46.390687 17432 raft_consensus.cc:504] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.390880 17432 leader_election.cc:288] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553), 92f0f405454541b381765455a60d913c (127.15.110.195:35539)
I20260927 18:43:46.391211 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b1f040bdd43f1a90a75d57c7416bb" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c" is_pre_election: true
I20260927 18:43:46.391219 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b1f040bdd43f1a90a75d57c7416bb" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b0db251eb4fedb9cd1e44f775daba" is_pre_election: true
I20260927 18:43:46.391309 17355 raft_consensus.cc:2454] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 0.
I20260927 18:43:46.391346 17075 raft_consensus.cc:2454] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 0.
I20260927 18:43:46.391469 17146 leader_election.cc:302] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f0f405454541b381765455a60d913c, a604b5d8b46249128f5701d4468faac8; no voters:
I20260927 18:43:46.391611 17432 raft_consensus.cc:2781] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:46.391669 17432 raft_consensus.cc:482] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:46.391691 17432 raft_consensus.cc:3037] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.392382 17432 raft_consensus.cc:504] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.392520 17432 leader_election.cc:288] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 election: Requested vote from peers f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553), 92f0f405454541b381765455a60d913c (127.15.110.195:35539)
I20260927 18:43:46.392680 17075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b1f040bdd43f1a90a75d57c7416bb" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f24b0db251eb4fedb9cd1e44f775daba"
I20260927 18:43:46.392715 17355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "002b1f040bdd43f1a90a75d57c7416bb" candidate_uuid: "a604b5d8b46249128f5701d4468faac8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92f0f405454541b381765455a60d913c"
I20260927 18:43:46.392774 17075 raft_consensus.cc:3037] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.392782 17355 raft_consensus.cc:3037] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:46.393566 17075 raft_consensus.cc:2454] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 1.
I20260927 18:43:46.393566 17355 raft_consensus.cc:2454] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a604b5d8b46249128f5701d4468faac8 in term 1.
I20260927 18:43:46.393792 17146 leader_election.cc:302] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92f0f405454541b381765455a60d913c, a604b5d8b46249128f5701d4468faac8; no voters:
I20260927 18:43:46.393889 17432 raft_consensus.cc:2781] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:46.393958 17432 raft_consensus.cc:686] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [term 1 LEADER]: Becoming Leader. State: Replica: a604b5d8b46249128f5701d4468faac8, State: Running, Role: LEADER
I20260927 18:43:46.394013 17432 consensus_queue.cc:237] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } }
I20260927 18:43:46.394311 17454 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: TxnStatusTablet state changed. Reason: New leader a604b5d8b46249128f5701d4468faac8. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.394408 17454 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:46.394586 17466 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:43:46.394727 17466 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:43:46.394922 16916 catalog_manager.cc:5495] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 reported cstate change: term changed from 0 to 1, leader changed from <none> to a604b5d8b46249128f5701d4468faac8 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:46.409065 17461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.406308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45233 (local address 127.0.0.1:55556)
0927 18:43:46.406391 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.406417 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.406466 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.406665 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.406666 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.406667 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:46.406731 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.406733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.407518 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.407519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.408108 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.408110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.408488 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.408489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.408559 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.408589 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.408880 (+ 291us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.408890 (+ 10us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.408942 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.408994 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:43:46.409057 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.406381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55556 (local address 127.15.110.194:45233)
0927 18:43:46.406443 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.406446 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.406456 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:46.406553 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.406554 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.406575 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.406656 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:46.406812 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.407426 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.408197 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.408409 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.408489 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.408697 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.408802 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.408882 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.408948 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.408973 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:43:46.410390 17355 raft_consensus.cc:1261] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Refusing update from remote peer a604b5d8b46249128f5701d4468faac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.410470 17075 raft_consensus.cc:1261] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Refusing update from remote peer a604b5d8b46249128f5701d4468faac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.410756 17432 consensus_queue.cc:1035] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.410914 17454 consensus_queue.cc:1035] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.411478 17430 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: TxnStatusTablet state changed. Reason: New leader a604b5d8b46249128f5701d4468faac8. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.411562 17430 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.411832 17431 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: TxnStatusTablet state changed. Reason: New leader a604b5d8b46249128f5701d4468faac8. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.411960 17431 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.412587 17432 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.412676 17432 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:46.412774 17430 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.412845 17430 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P 92f0f405454541b381765455a60d913c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.412949 17439 mvcc.cc:205] Tried to move back new op lower bound from 7334029829774704640 to 7334029829710307328. Current Snapshot: MvccSnapshot[applied={T|T < 7334029829774704640}]
I20260927 18:43:46.413964 17432 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.414054 17432 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P a604b5d8b46249128f5701d4468faac8: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:46.414314 17431 tablet_replica.cc:439] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a604b5d8b46249128f5701d4468faac8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 } } peers { permanent_uuid: "a604b5d8b46249128f5701d4468faac8" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45233 } } peers { permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 } } }
I20260927 18:43:46.414403 17431 tablet_replica.cc:442] T 002b1f040bdd43f1a90a75d57c7416bb P f24b0db251eb4fedb9cd1e44f775daba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:46.417121 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.414891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55570 (local address 127.15.110.194:45233)
0927 18:43:46.414926 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.414928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.414937 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:46.414958 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.414958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.414968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.415023 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:46.415123 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.415646 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.416412 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.416623 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.416697 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.416858 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.416891 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.416960 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.417020 (+ 60us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.417038 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:43:46.422932 17423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.419976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:58977 (local address 127.15.110.194:45233)
0927 18:43:46.420017 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.420020 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.420187 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:43:46.420247 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.420248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.420256 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.420300 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:46.420461 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.420925 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.422188 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.422362 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.422430 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.422692 (+ 262us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.422725 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.422778 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.422861 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.422878 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:46.423062 17473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.419889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45233 (local address 127.15.110.194:58977)
0927 18:43:46.420103 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:46.420115 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:46.420183 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:46.420305 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:46.420306 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:46.420308 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:46.420375 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.420377 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.421013 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.421014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.422090 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:46.422093 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.422457 (+ 364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.422458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:46.422543 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.422564 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:46.422797 (+ 233us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:46.422798 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:43:46.422859 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:43:46.422916 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":98,"threads_started":1}
I20260927 18:43:46.425565 17075 raft_consensus.cc:1261] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Refusing update from remote peer a604b5d8b46249128f5701d4468faac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.425721 17355 raft_consensus.cc:1261] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Refusing update from remote peer a604b5d8b46249128f5701d4468faac8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:46.425794 17432 consensus_queue.cc:1035] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f24b0db251eb4fedb9cd1e44f775daba" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.425917 17454 consensus_queue.cc:1035] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92f0f405454541b381765455a60d913c" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:46.441540 16916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:46.441648 16916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60864:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:46.442310 16916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P acdb310264a24683b8d9361ef2231a16: Sending DeleteTablet for 3 replicas of tablet 95ba2471503f4279a8e0cb0a9775b966
I20260927 18:43:46.442585 17195 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55550
I20260927 18:43:46.442615 17055 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:56578
I20260927 18:43:46.442679 17449 tablet_replica.cc:330] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: stopping tablet replica
I20260927 18:43:46.442667 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 95ba2471503f4279a8e0cb0a9775b966 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:46 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:40532
I20260927 18:43:46.442714 17447 tablet_replica.cc:330] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: stopping tablet replica
I20260927 18:43:46.442723 17449 raft_consensus.cc:2229] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:46.442760 17447 raft_consensus.cc:2229] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:46.442763 17448 tablet_replica.cc:330] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: stopping tablet replica
I20260927 18:43:46.442797 17447 raft_consensus.cc:2258] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.442808 17449 raft_consensus.cc:2258] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.442813 17448 raft_consensus.cc:2229] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:46.442852 17448 raft_consensus.cc:2258] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:46.443257 17449 ts_tablet_manager.cc:1905] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.443404 17448 ts_tablet_manager.cc:1905] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.444809 17449 ts_tablet_manager.cc:1918] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:46.444859 17449 log.cc:1196] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/95ba2471503f4279a8e0cb0a9775b966
I20260927 18:43:46.444881 17448 ts_tablet_manager.cc:1918] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:46.444936 17448 log.cc:1196] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/95ba2471503f4279a8e0cb0a9775b966
I20260927 18:43:46.445075 17449 ts_tablet_manager.cc:1939] T 95ba2471503f4279a8e0cb0a9775b966 P a604b5d8b46249128f5701d4468faac8: Deleting consensus metadata
I20260927 18:43:46.445187 17448 ts_tablet_manager.cc:1939] T 95ba2471503f4279a8e0cb0a9775b966 P 92f0f405454541b381765455a60d913c: Deleting consensus metadata
I20260927 18:43:46.445525 16902 catalog_manager.cc:4841] TS 92f0f405454541b381765455a60d913c (127.15.110.195:35539): tablet 95ba2471503f4279a8e0cb0a9775b966 (table test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]) successfully deleted
I20260927 18:43:46.445601 16904 catalog_manager.cc:4841] TS a604b5d8b46249128f5701d4468faac8 (127.15.110.194:45233): tablet 95ba2471503f4279a8e0cb0a9775b966 (table test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]) successfully deleted
I20260927 18:43:46.447423 17447 ts_tablet_manager.cc:1905] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:46.449065 17447 ts_tablet_manager.cc:1918] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:46.449129 17447 log.cc:1196] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/95ba2471503f4279a8e0cb0a9775b966
I20260927 18:43:46.449378 17447 ts_tablet_manager.cc:1939] T 95ba2471503f4279a8e0cb0a9775b966 P f24b0db251eb4fedb9cd1e44f775daba: Deleting consensus metadata
I20260927 18:43:46.449759 16903 catalog_manager.cc:4841] TS f24b0db251eb4fedb9cd1e44f775daba (127.15.110.193:43553): tablet 95ba2471503f4279a8e0cb0a9775b966 (table test-table [id=cfa4e5031b814f82b8bdfb6be6aa58b9]) successfully deleted
Sep 27 18:43:46 dist-test-slave-vs05 krb5kdc[16873](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534625, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:46.454250 17421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.444816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56600 (local address 127.15.110.193:43553)
0927 18:43:46.444859 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.444861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.444982 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:43:46.445096 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.445098 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.445119 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.445151 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:46.445969 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.446745 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.447813 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.448145 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.448218 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.451326 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:46.451334 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:46.451335 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:46.451349 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:46.453034 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.453431 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.453435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.453436 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.453497 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.453708 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.453709 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.453709 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.453852 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:46.454006 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.454071 (+ 65us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.454152 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 18:43:46.454806 17115 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56600
I20260927 18:43:46.462641 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:46.455892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60876 (local address 127.15.110.254:46083)
0927 18:43:46.455955 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:46.455957 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:46.456101 (+ 144us) server_negotiation.cc:407] Connection header received
0927 18:43:46.456176 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:46.456177 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:46.456186 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:46.456232 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:46.456845 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.457398 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.458409 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:46.458600 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:46.458656 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:46.460019 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:46.460021 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:46.460021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:46.460027 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:46.461483 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.461869 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.461869 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.461870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.461923 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:46.462146 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:46.462146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:46.462147 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:46.462252 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:46.462329 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:46.462468 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:43:46.462526 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260927 18:43:46.462983 16916 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60876
I20260927 18:43:46.463730 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16990
I20260927 18:43:46.471108 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17130
I20260927 18:43:46.476693 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17270
I20260927 18:43:46.481184 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 16886
2026-09-27T18:43:46Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2326 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:46 dist-test-slave-vs05 krb5kdc[17484](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:46 dist-test-slave-vs05 krb5kdc[17484](info): set up 2 sockets
Sep 27 18:43:46 dist-test-slave-vs05 krb5kdc[17484](info): commencing operation
krb5kdc: starting...
W20260927 18:43:47.546752 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:47 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:47.665534 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40485
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:40485
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:47.746737 17497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:47.746939 17497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:47.746961 17497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:47.748150 17497 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:47.748201 17497 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:43:47.748216 17497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:47.748229 17497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:47.748291 17497 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:47.748308 17497 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:47.749782 17497 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:40485
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40485
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:47.750077 17497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:47.750344 17497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:47.755401 17505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:47.755424 17502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:47.755442 17503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:47.755879 17497 server_base.cc:1034] running on GCE node
I20260927 18:43:47.756129 17497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:47.756419 17497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:47.757606 17497 hybrid_clock.cc:648] HybridClock initialized: now 1790534627757587 us; error 29 us; skew 500 ppm
Sep 27 18:43:47 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:47.760037 17497 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:47.760455 17497 webserver.cc:460] Webserver started at http://127.15.110.254:39113/ using document root <none> and password file <none>
I20260927 18:43:47.760720 17497 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:47.760802 17497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:47.760951 17497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:47.762033 17497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "f7ac8d8af0e81205f5b1a272326aa06b"
server_key_iv: "ccf431706f86db55505ce7939cb5fee9"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.762399 17497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "f7ac8d8af0e81205f5b1a272326aa06b"
server_key_iv: "ccf431706f86db55505ce7939cb5fee9"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.763760 17497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:47.764480 17512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:47.764650 17497 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:47.764756 17497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "f7ac8d8af0e81205f5b1a272326aa06b"
server_key_iv: "ccf431706f86db55505ce7939cb5fee9"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.764832 17497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:47.772701 17497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:47.774438 17497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:47.774610 17497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:47.779451 17497 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:40485
I20260927 18:43:47.779510 17574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:40485 every 8 connection(s)
I20260927 18:43:47.779827 17497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:47.780356 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17497
I20260927 18:43:47.780434 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:47.780457 17575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:47.780627 15803 external_mini_cluster.cc:1428] Setting key dd86a7a0dac2382fdf9b885818408a41
I20260927 18:43:47.783310 17575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Bootstrap starting.
I20260927 18:43:47.783912 17575 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:47.784186 17575 log.cc:824] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:47.784881 17575 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: No bootstrap required, opened a new log
Sep 27 18:43:47 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:47.787066 17575 raft_consensus.cc:348] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } }
I20260927 18:43:47.787271 17575 raft_consensus.cc:374] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:47.787312 17575 raft_consensus.cc:729] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0d46c74c12a46a98f8496f9f5dbeeb8, State: Initialized, Role: FOLLOWER
I20260927 18:43:47.787454 17575 consensus_queue.cc:260] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } }
I20260927 18:43:47.787550 17575 raft_consensus.cc:388] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:47.787583 17575 raft_consensus.cc:482] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:47.787638 17575 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:47.788393 17575 raft_consensus.cc:504] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } }
I20260927 18:43:47.788513 17575 leader_election.cc:302] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e0d46c74c12a46a98f8496f9f5dbeeb8; no voters:
I20260927 18:43:47.788691 17575 leader_election.cc:288] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:47.788751 17580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:47.788952 17580 raft_consensus.cc:686] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [term 1 LEADER]: Becoming Leader. State: Replica: e0d46c74c12a46a98f8496f9f5dbeeb8, State: Running, Role: LEADER
I20260927 18:43:47.788977 17575 sys_catalog.cc:564] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:47.789093 17580 consensus_queue.cc:237] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } }
I20260927 18:43:47.789469 17580 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e0d46c74c12a46a98f8496f9f5dbeeb8. Latest consensus state: current_term: 1 leader_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } } }
I20260927 18:43:47.789491 17581 sys_catalog.cc:455] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e0d46c74c12a46a98f8496f9f5dbeeb8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40485 } } }
I20260927 18:43:47.789626 17580 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:47.789634 17581 sys_catalog.cc:458] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:47.789933 17589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:47.790625 17589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:47.791712 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:47.781181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.15.110.254:40485)
0927 18:43:47.781371 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:47.781379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:47.781403 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:43:47.781835 (+ 432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:47.781841 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:47.781891 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:47.782040 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:47.782702 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.783717 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.784598 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.785044 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.785171 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:47.787820 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:47.787824 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:47.787824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:47.787837 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:47.789865 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:47.790324 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:47.790327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:47.790327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:47.790448 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:47.790743 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:47.790745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:47.790745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:47.791047 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:47.791236 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:47.791373 (+ 137us) server_negotiation.cc:299] Negotiation successful
0927 18:43:47.791511 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":77,"threads_started":1}
I20260927 18:43:47.795656 17589 catalog_manager.cc:1347] Generated new cluster ID: b58bdd92842140ffa042ec23c580313a
I20260927 18:43:47.795710 17589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:47.806521 17589 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:47.807107 17589 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:47.811506 17589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Generated new TSK 0
I20260927 18:43:47.811715 17589 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:47.842067 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40485
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:47.942526 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:47.942734 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:47.942759 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:47.943991 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:47.944039 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:47.944060 17601 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:43:47.944073 17601 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:47.944090 17601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:47.945573 17601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:47.945824 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:47.946054 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:47.951169 17606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:47.951229 17609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:47.951361 17601 server_base.cc:1034] running on GCE node
W20260927 18:43:47.951169 17607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:47.951704 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:47.951944 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:47.953100 17601 hybrid_clock.cc:648] HybridClock initialized: now 1790534627953103 us; error 37 us; skew 500 ppm
Sep 27 18:43:47 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:47.955631 17601 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:47.956068 17601 webserver.cc:460] Webserver started at http://127.15.110.193:34491/ using document root <none> and password file <none>
I20260927 18:43:47.956264 17601 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:47.956308 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:47.956404 17601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:47.957561 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "e753c37be4b39b88b26a7e2343081503"
server_key_iv: "8b28525f377aba78aa650cb0b3f48d72"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.957895 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "e753c37be4b39b88b26a7e2343081503"
server_key_iv: "8b28525f377aba78aa650cb0b3f48d72"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.959293 17601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:43:47.960053 17616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:47.960216 17601 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:47.960300 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
format_stamp: "Formatted at 2026-09-27 18:43:47 on dist-test-slave-vs05"
server_key: "e753c37be4b39b88b26a7e2343081503"
server_key_iv: "8b28525f377aba78aa650cb0b3f48d72"
server_key_version: "encryptionkey@0"
I20260927 18:43:47.960354 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:47.981004 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:47.982879 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:47.982992 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:47.983486 17601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:47.983553 17601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:47.983598 17601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:47.983628 17601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:47.990890 17601 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39455
I20260927 18:43:47.991312 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:47.991349 17731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39455 every 8 connection(s)
Sep 27 18:43:47 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:47.994406 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:47.984125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:57231 (local address 127.15.110.254:40485)
0927 18:43:47.984194 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:47.984196 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:47.984471 (+ 275us) server_negotiation.cc:407] Connection header received
0927 18:43:47.985102 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:47.985104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:47.985118 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:47.985215 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:47.986173 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.987164 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.988925 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.989159 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.989231 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:47.992109 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:47.992110 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:47.992120 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:47.992127 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:47.993248 (+ 1121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:47.993614 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:47.993615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:47.993615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:47.993665 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:47.993873 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:47.993873 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:47.993873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:47.993980 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:47.994036 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:47.994276 (+ 240us) server_negotiation.cc:299] Negotiation successful
0927 18:43:47.994342 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:43:47.994637 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:47.984105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.193:57231)
0927 18:43:47.984294 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:47.984332 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:47.984987 (+ 655us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:47.985342 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:47.985348 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:47.985779 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:47.986078 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:47.986087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.987305 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.987307 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:47.988790 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:47.988793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.989263 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.989264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:47.989427 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:47.990119 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:47.990129 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:47.992015 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:47.993323 (+ 1308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:47.993329 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:47.993330 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:47.993546 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:47.993722 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:47.993723 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:47.993723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:47.993818 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:47.994034 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:47.994038 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:47.994175 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:43:47.994263 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:43:47.994380 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260927 18:43:47.995121 17732 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40485
I20260927 18:43:47.995247 17732 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:47.995448 17732 heartbeater.cc:511] Master 127.15.110.254:40485 requested a full tablet report, sending...
I20260927 18:43:47.995977 17528 ts_manager.cc:194] Registered new tserver with Master: 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455)
I20260927 18:43:47.996923 17528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:57231
I20260927 18:43:47.997745 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17601
I20260927 18:43:47.997849 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:47.998030 15803 external_mini_cluster.cc:1428] Setting key cd79e951ce99b1a29840540969223f29
I20260927 18:43:48.002508 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:47.996290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:40065 (local address 127.15.110.254:40485)
0927 18:43:47.996328 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:47.996329 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:47.996444 (+ 115us) server_negotiation.cc:407] Connection header received
0927 18:43:47.996509 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:47.996509 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:47.996517 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:47.996557 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:47.996877 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.997646 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.998599 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.998876 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.998931 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:47.999968 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:47.999970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:47.999970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:47.999975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.001334 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.001691 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.001692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.001693 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.001752 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.001972 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.001973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.001974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.002079 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.002166 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.002322 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.002411 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:43:48.002509 17738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:47.996215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.193:40065)
0927 18:43:47.996387 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:47.996398 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:47.996456 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:47.996578 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:47.996579 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:47.996760 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:47.996813 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:47.996815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.997763 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.997765 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:47.998524 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:47.998526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:47.998960 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:47.998961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:47.999046 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:47.999464 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:47.999470 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:47.999885 (+ 415us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.001436 (+ 1551us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.001438 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.001438 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.001618 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.001820 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.001821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.001821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.001907 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.002161 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.002162 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.002208 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.002296 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.002413 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:48.023986 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40485
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:48.103914 17741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:48.104146 17741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:48.104185 17741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:48.105836 17741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:48.105878 17741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:48.105901 17741 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:43:48.105914 17741 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:48.105969 17741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:48.107438 17741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:48.107679 17741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:48.107910 17741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:48.112794 17747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:48.112789 17746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:48.112910 17741 server_base.cc:1034] running on GCE node
W20260927 18:43:48.112984 17749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:48.113204 17741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:48.113526 17741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:48.114704 17741 hybrid_clock.cc:648] HybridClock initialized: now 1790534628114684 us; error 30 us; skew 500 ppm
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:48.117105 17741 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:48.117583 17741 webserver.cc:460] Webserver started at http://127.15.110.194:44615/ using document root <none> and password file <none>
I20260927 18:43:48.117815 17741 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:48.117882 17741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:48.118016 17741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:48.119048 17741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "a4fe2952158741158f89045b1a1b5736"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "3f9fa35fe2d9b40a7843a4af103ef0ac"
server_key_iv: "b24d0a9f24827d80ddf3091ae12ab732"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.119458 17741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "a4fe2952158741158f89045b1a1b5736"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "3f9fa35fe2d9b40a7843a4af103ef0ac"
server_key_iv: "b24d0a9f24827d80ddf3091ae12ab732"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.120782 17741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:48.121488 17756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.121682 17741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:48.121783 17741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "a4fe2952158741158f89045b1a1b5736"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "3f9fa35fe2d9b40a7843a4af103ef0ac"
server_key_iv: "b24d0a9f24827d80ddf3091ae12ab732"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.121861 17741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:48.139745 17741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:48.141583 17741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:48.141752 17741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:48.142405 17741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:48.142463 17741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.142508 17741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:48.142522 17741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:48.149801 17741 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:43847
I20260927 18:43:48.149909 17871 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:43847 every 8 connection(s)
I20260927 18:43:48.150195 17741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:48.152279 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.142876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:60063 (local address 127.15.110.254:40485)
0927 18:43:48.142949 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.142951 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.143191 (+ 240us) server_negotiation.cc:407] Connection header received
0927 18:43:48.143708 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.143710 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.143725 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.143804 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.144537 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.145437 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.146450 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.146679 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.146766 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.149462 (+ 2696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.149464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.149464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.149471 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.151013 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.151505 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.151506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.151507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.151562 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.151773 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.151773 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.151774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.151870 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.151917 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.152156 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.152218 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:43:48.152417 17765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.142845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.194:60063)
0927 18:43:48.143032 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.143070 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.143610 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.143878 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.143881 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.144185 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:48.144374 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.144379 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.145537 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.145539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.146355 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.146356 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.146777 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.146778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.146900 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.147426 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:48.147436 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:48.149339 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.151085 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.151090 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.151091 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.151414 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.151631 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.151631 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.151632 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.151729 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.151934 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.151939 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.152057 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.152134 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.152246 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":111,"threads_started":1}
I20260927 18:43:48.152973 17872 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40485
I20260927 18:43:48.153065 17872 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:48.153250 17872 heartbeater.cc:511] Master 127.15.110.254:40485 requested a full tablet report, sending...
I20260927 18:43:48.153668 17528 ts_manager.cc:194] Registered new tserver with Master: a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847)
I20260927 18:43:48.154120 17528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:60063
I20260927 18:43:48.158845 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17741
I20260927 18:43:48.158955 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:48.159166 15803 external_mini_cluster.cc:1428] Setting key 15b58975c8f39e2052698e853a14da86
I20260927 18:43:48.159197 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.154017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.194:55139)
0927 18:43:48.154132 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.154142 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.154190 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.154378 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.154379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.154614 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:48.154669 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.154671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.155436 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.155437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.155948 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.155951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.156255 (+ 304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.156256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.156308 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.156748 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:48.156753 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:48.157022 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.158262 (+ 1240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.158262 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.158263 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.158416 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.158569 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.158569 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.158570 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.158610 (+ 40us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.158842 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.158843 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.158892 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.158975 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.159077 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":66,"threads_started":1}
I20260927 18:43:48.159241 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.154081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:55139 (local address 127.15.110.254:40485)
0927 18:43:48.154258 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.154260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.154268 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:48.154287 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.154287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.154297 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.154349 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.154751 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.155356 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.156017 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.156198 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.156246 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.157090 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.157091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.157091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.157096 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.158194 (+ 1098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.158486 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.158487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.158487 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.158533 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.158659 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.158660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.158660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.158779 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.158826 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.159045 (+ 219us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.159130 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160}
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:48.184158 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40485
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37365
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:48.264792 17881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:48.264981 17881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:48.265005 17881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:48.266063 17881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:48.266100 17881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:48.266126 17881 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260927 18:43:48.266139 17881 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:43:48.266155 17881 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:48.267588 17881 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37365
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:40485
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:48.267892 17881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:48.268123 17881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:48.273147 17887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:48.273164 17889 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:48.273216 17881 server_base.cc:1034] running on GCE node
W20260927 18:43:48.273145 17886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:48.273552 17881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:48.273804 17881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:48.274948 17881 hybrid_clock.cc:648] HybridClock initialized: now 1790534628274932 us; error 33 us; skew 500 ppm
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:48.277486 17881 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:48.277958 17881 webserver.cc:460] Webserver started at http://127.15.110.195:33351/ using document root <none> and password file <none>
I20260927 18:43:48.278191 17881 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:48.278236 17881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:48.278383 17881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:48.279590 17881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "efd44071853246a4aa9410a8d822e917"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "af371bf6576bd9a3479f581b92f4ed95"
server_key_iv: "2f394571fa5028d0c0f3f215ce5abf06"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.279951 17881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "efd44071853246a4aa9410a8d822e917"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "af371bf6576bd9a3479f581b92f4ed95"
server_key_iv: "2f394571fa5028d0c0f3f215ce5abf06"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.281234 17881 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:43:48.281941 17896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.282130 17881 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:48.282234 17881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "efd44071853246a4aa9410a8d822e917"
format_stamp: "Formatted at 2026-09-27 18:43:48 on dist-test-slave-vs05"
server_key: "af371bf6576bd9a3479f581b92f4ed95"
server_key_iv: "2f394571fa5028d0c0f3f215ce5abf06"
server_key_version: "encryptionkey@0"
I20260927 18:43:48.282321 17881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:48.297997 17881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:48.299887 17881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:48.300045 17881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:48.300586 17881 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:48.300647 17881 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.300705 17881 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:48.300735 17881 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:48.308141 17881 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:35783
I20260927 18:43:48.308210 18011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:35783 every 8 connection(s)
I20260927 18:43:48.308535 17881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:48.308990 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 17881
I20260927 18:43:48.309091 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:48.309269 15803 external_mini_cluster.cc:1428] Setting key 851d31dc7d41f3896db57231b8dec7bf
I20260927 18:43:48.310964 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.300946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:53503 (local address 127.15.110.254:40485)
0927 18:43:48.301014 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.301017 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.301260 (+ 243us) server_negotiation.cc:407] Connection header received
0927 18:43:48.301728 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.301730 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.301744 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.301831 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.302449 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.303335 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.304436 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.304658 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.304746 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.307885 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.307886 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.307887 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.307895 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.309551 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.310030 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.310031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.310032 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.310092 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.310333 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.310334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.310334 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.310451 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.310531 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.310795 (+ 264us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.310874 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:43:48.311151 17905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.300951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.195:53503)
0927 18:43:48.301126 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.301165 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.301632 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.301880 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.301883 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.302118 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:48.302347 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.302351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.303438 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.303439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.304339 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.304342 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.304757 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.304758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.304892 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.305375 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:48.305394 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:48.307781 (+ 2387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.309640 (+ 1859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.309646 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.309648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.309952 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.310151 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.310151 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.310152 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.310260 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.310522 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.310528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.310665 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.310776 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.310926 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
I20260927 18:43:48.311600 18012 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40485
I20260927 18:43:48.311697 18012 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:48.311873 18012 heartbeater.cc:511] Master 127.15.110.254:40485 requested a full tablet report, sending...
I20260927 18:43:48.312279 17528 ts_manager.cc:194] Registered new tserver with Master: efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783)
I20260927 18:43:48.312779 17528 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:53503
I20260927 18:43:48.317951 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.312726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:38311 (local address 127.15.110.254:40485)
0927 18:43:48.312765 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.312768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.312837 (+ 69us) server_negotiation.cc:407] Connection header received
0927 18:43:48.312898 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.312899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.312908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.312962 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.313273 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.313744 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.314447 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.314659 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.314746 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.315594 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.315596 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.315596 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.315602 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.316843 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.317185 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.317186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.317186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.317238 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.317438 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.317439 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.317439 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.317546 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.317639 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.317791 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.317863 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:48.317996 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.312673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.15.110.195:38311)
0927 18:43:48.312785 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.312795 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.312852 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.312979 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.312980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.313151 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:48.313200 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.313201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.313835 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.313836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.314365 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.314366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.314748 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.314749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.314828 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.315245 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:48.315249 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:48.315503 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.316933 (+ 1430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.316934 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.316934 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.317114 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.317312 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.317313 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.317313 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.317373 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.317634 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.317635 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.317680 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.317764 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.317900 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89,"thread_start_us":61,"threads_started":1}
I20260927 18:43:48.320879 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534628, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:48.343168 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.336351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.15.110.254:40485)
0927 18:43:48.336425 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.336427 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.336437 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:48.336470 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.336471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.336479 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.336543 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.337141 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.337626 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.338376 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.338567 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.338641 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.340846 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.340847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.340848 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.340855 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.342001 (+ 1146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.342354 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.342355 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.342356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.342416 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.342624 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.342625 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.342625 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.342735 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.342835 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.342961 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.343038 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:43:48.344835 17528 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:48.346393 17528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:48.353030 18030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.348787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.0.0.1:52434)
0927 18:43:48.349041 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.349109 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.349177 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.349597 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.349598 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.349600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.349655 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.349657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.350633 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.350634 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.351762 (+ 1128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.351765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.352692 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.352693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.352784 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.352799 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.352880 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.352934 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":55,"threads_started":1}
I20260927 18:43:48.353051 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.349181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.15.110.195:35783)
0927 18:43:48.349357 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.349362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.349373 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:48.349420 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.349422 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.349439 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.349513 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.349737 (+ 224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.350533 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.351848 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.352595 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.352682 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.352737 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.352915 (+ 178us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.352953 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":84,"threads_started":1}
I20260927 18:43:48.353485 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.348978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33012 (local address 127.15.110.194:43847)
0927 18:43:48.349152 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.349157 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.349386 (+ 229us) server_negotiation.cc:407] Connection header received
0927 18:43:48.349439 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.349442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.349459 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.349560 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.349738 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.350860 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.352198 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.352960 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.353021 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.353091 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.353304 (+ 213us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.353343 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":83,"threads_started":1}
I20260927 18:43:48.353523 18029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.348799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39455 (local address 127.0.0.1:55162)
0927 18:43:48.349373 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.349386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.349453 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.349878 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.349879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.349880 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.349929 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.349931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.351160 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.351161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.352246 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.352249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.353248 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.353250 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.353320 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.353325 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.353413 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.353454 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":94,"threads_started":1}
I20260927 18:43:48.353790 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.349066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.15.110.193:39455)
0927 18:43:48.349243 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.349249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.349446 (+ 197us) server_negotiation.cc:407] Connection header received
0927 18:43:48.349716 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.349718 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.349734 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.349809 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.350017 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.351078 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.352330 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.353106 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.353241 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.353299 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.353431 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.353469 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":70,"threads_started":1}
I20260927 18:43:48.353865 18031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.348787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:43847 (local address 127.0.0.1:33012)
0927 18:43:48.349304 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.349314 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.349366 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.349566 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.349569 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.349571 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.349647 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.349649 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.350962 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.350963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.352104 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.352107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.353126 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.353128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.353213 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.353216 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.353746 (+ 530us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.353796 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":20,"thread_start_us":77,"threads_started":1}
I20260927 18:43:48.354127 17946 tablet_service.cc:1467] Processing CreateTablet for tablet 90535227e7bd4b75b03c65ac4110f76d (DEFAULT_TABLE table=test-table [id=2815ae394e414a038f293d786d92d74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.354516 17946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90535227e7bd4b75b03c65ac4110f76d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.354573 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 90535227e7bd4b75b03c65ac4110f76d (DEFAULT_TABLE table=test-table [id=2815ae394e414a038f293d786d92d74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.354591 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 90535227e7bd4b75b03c65ac4110f76d (DEFAULT_TABLE table=test-table [id=2815ae394e414a038f293d786d92d74c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.354882 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90535227e7bd4b75b03c65ac4110f76d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.354933 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90535227e7bd4b75b03c65ac4110f76d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.357883 18035 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Bootstrap starting.
I20260927 18:43:48.358489 18035 tablet_bootstrap.cc:654] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.358502 18037 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Bootstrap starting.
I20260927 18:43:48.358594 18036 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Bootstrap starting.
I20260927 18:43:48.358909 18035 log.cc:824] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:48.359148 18037 tablet_bootstrap.cc:654] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.359440 18036 tablet_bootstrap.cc:654] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.359498 18037 log.cc:824] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:48.359743 18035 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: No bootstrap required, opened a new log
I20260927 18:43:48.359771 18036 log.cc:824] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:48.359808 18035 ts_tablet_manager.cc:1397] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:48.360252 18037 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: No bootstrap required, opened a new log
I20260927 18:43:48.360320 18037 ts_tablet_manager.cc:1397] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:48.360517 18036 tablet_bootstrap.cc:492] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: No bootstrap required, opened a new log
I20260927 18:43:48.360600 18036 ts_tablet_manager.cc:1397] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:43:48.361997 18036 raft_consensus.cc:348] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362001 18035 raft_consensus.cc:348] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362154 18036 raft_consensus.cc:374] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.362160 18035 raft_consensus.cc:374] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.362198 18035 raft_consensus.cc:729] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.362198 18036 raft_consensus.cc:729] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd44071853246a4aa9410a8d822e917, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.362310 18036 consensus_queue.cc:260] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362317 18035 consensus_queue.cc:260] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362238 18037 raft_consensus.cc:348] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362434 18037 raft_consensus.cc:374] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.362478 18037 raft_consensus.cc:729] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.362581 18035 ts_tablet_manager.cc:1428] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:43:48.362591 18036 ts_tablet_manager.cc:1428] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:43:48.362589 18037 consensus_queue.cc:260] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.362645 17872 heartbeater.cc:503] Master 127.15.110.254:40485 was elected leader, sending a full tablet report...
I20260927 18:43:48.362823 18037 ts_tablet_manager.cc:1428] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:43:48.362836 17732 heartbeater.cc:503] Master 127.15.110.254:40485 was elected leader, sending a full tablet report...
I20260927 18:43:48.362896 18012 heartbeater.cc:503] Master 127.15.110.254:40485 was elected leader, sending a full tablet report...
W20260927 18:43:48.400838 17873 tablet.cc:2369] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:48.474880 18042 raft_consensus.cc:482] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:48.474992 18042 raft_consensus.cc:504] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.475440 18042 leader_election.cc:288] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847), 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455)
I20260927 18:43:48.478358 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.475709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:54105 (local address 127.15.110.193:39455)
0927 18:43:48.475757 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.475760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.475768 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:43:48.475803 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.475804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.475815 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.475893 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.476082 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.476618 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.477627 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.477994 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.478070 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.478079 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.478244 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.478283 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:43:48.478384 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.475716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:52939 (local address 127.15.110.194:43847)
0927 18:43:48.475757 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.475759 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.475928 (+ 169us) server_negotiation.cc:407] Connection header received
0927 18:43:48.475974 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.475976 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.475987 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.476041 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.476231 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.476710 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.477632 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.477998 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.478086 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.478095 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.478274 (+ 179us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.478316 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:48.478395 17905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.475634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39455 (local address 127.15.110.195:54105)
0927 18:43:48.475686 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.475698 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.475744 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.475924 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.475925 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.475927 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.476008 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.476010 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.476698 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.476699 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.477513 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.477515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.478077 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.478078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.478163 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.478166 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.478237 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.478275 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":30}
I20260927 18:43:48.478408 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.475640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:43847 (local address 127.15.110.195:52939)
0927 18:43:48.475848 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.475859 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.475917 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.476067 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.476068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.476096 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.476162 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.476163 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.476784 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.476785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.477521 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.477523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.478108 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.478109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.478210 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.478214 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.478268 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.478304 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":54,"thread_start_us":93,"threads_started":1}
I20260927 18:43:48.478802 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90535227e7bd4b75b03c65ac4110f76d" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" is_pre_election: true
I20260927 18:43:48.478973 17686 raft_consensus.cc:2454] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:48.479007 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90535227e7bd4b75b03c65ac4110f76d" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736" is_pre_election: true
I20260927 18:43:48.479127 17826 raft_consensus.cc:2454] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:48.479373 17899 leader_election.cc:302] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fe2952158741158f89045b1a1b5736, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:48.479508 18042 raft_consensus.cc:2781] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:48.479583 18042 raft_consensus.cc:482] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:48.479607 18042 raft_consensus.cc:3037] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.480299 18042 raft_consensus.cc:504] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.480439 18042 leader_election.cc:288] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Requested vote from peers a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847), 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455)
I20260927 18:43:48.480602 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90535227e7bd4b75b03c65ac4110f76d" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736"
I20260927 18:43:48.480609 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90535227e7bd4b75b03c65ac4110f76d" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
I20260927 18:43:48.480681 17826 raft_consensus.cc:3037] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.480681 17686 raft_consensus.cc:3037] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.481371 17826 raft_consensus.cc:2454] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:48.481379 17686 raft_consensus.cc:2454] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:48.481554 17899 leader_election.cc:302] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fe2952158741158f89045b1a1b5736, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:48.481679 18042 raft_consensus.cc:2781] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:48.481848 18042 raft_consensus.cc:686] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 1 LEADER]: Becoming Leader. State: Replica: efd44071853246a4aa9410a8d822e917, State: Running, Role: LEADER
I20260927 18:43:48.481956 18042 consensus_queue.cc:237] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } }
I20260927 18:43:48.482812 17527 catalog_manager.cc:5495] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd44071853246a4aa9410a8d822e917 (127.15.110.195). New cstate: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } health_report { overall_health: UNKNOWN } } }
W20260927 18:43:48.492125 17733 tablet.cc:2369] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:43:48 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534627, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:48.526774 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.519952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40485 (local address 127.0.0.1:50512)
0927 18:43:48.520133 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.520143 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.520231 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.520378 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.520378 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.520702 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:48.520748 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.520751 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.521389 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.521391 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.521918 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.521920 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.522285 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.522286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.522343 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.522850 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:48.522854 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:48.524273 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:48.525818 (+ 1545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.525825 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.525827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.526040 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.526215 (+ 175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:48.526216 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:48.526216 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:48.526274 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:48.526473 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:48.526479 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:48.526522 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.526573 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.526676 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260927 18:43:48.526844 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.520012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50512 (local address 127.15.110.254:40485)
0927 18:43:48.520069 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.520071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.520210 (+ 139us) server_negotiation.cc:407] Connection header received
0927 18:43:48.520299 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.520300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.520313 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.520373 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:48.520828 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.521313 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.522011 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.522194 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.522273 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.524353 (+ 2080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:48.524355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:48.524355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:48.524361 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:48.525699 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.526105 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.526106 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.526107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.526155 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:48.526323 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:48.526324 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:48.526324 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:48.526422 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:48.526469 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.526621 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.526720 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":37,"server-negotiator.queue_time_us":31}
I20260927 18:43:48.528800 17527 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:50512:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:43:48.529031 17527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:48.530553 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 391553716723429088b5e2cb66a5e5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c88e0fc443ca43d4a89730650691c6c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:48.530527 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 391553716723429088b5e2cb66a5e5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c88e0fc443ca43d4a89730650691c6c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:48.530718 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 391553716723429088b5e2cb66a5e5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.530733 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 391553716723429088b5e2cb66a5e5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.530819 17946 tablet_service.cc:1467] Processing CreateTablet for tablet 391553716723429088b5e2cb66a5e5bf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c88e0fc443ca43d4a89730650691c6c8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:43:48.530956 17946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 391553716723429088b5e2cb66a5e5bf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.532346 18037 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: Bootstrap starting.
I20260927 18:43:48.532382 18036 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: Bootstrap starting.
I20260927 18:43:48.532755 18037 tablet_bootstrap.cc:654] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.532763 18036 tablet_bootstrap.cc:654] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.533041 18035 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: Bootstrap starting.
I20260927 18:43:48.533358 18036 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: No bootstrap required, opened a new log
I20260927 18:43:48.533393 18036 ts_tablet_manager.cc:1397] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:48.533529 18036 raft_consensus.cc:348] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.533622 18036 raft_consensus.cc:374] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.533638 18035 tablet_bootstrap.cc:654] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.533661 18036 raft_consensus.cc:729] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd44071853246a4aa9410a8d822e917, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.533716 18036 consensus_queue.cc:260] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.533813 18036 ts_tablet_manager.cc:1428] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:48.533918 18042 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.533986 18042 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.534241 18037 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: No bootstrap required, opened a new log
I20260927 18:43:48.534297 18037 ts_tablet_manager.cc:1397] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:48.534447 18037 raft_consensus.cc:348] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.534524 18037 raft_consensus.cc:374] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.534548 18037 raft_consensus.cc:729] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.534621 18037 consensus_queue.cc:260] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.534731 18037 ts_tablet_manager.cc:1428] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.534811 18043 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.534919 18043 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.535005 18035 tablet_bootstrap.cc:492] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: No bootstrap required, opened a new log
I20260927 18:43:48.535058 18035 ts_tablet_manager.cc:1397] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:48.535214 18035 raft_consensus.cc:348] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.535287 18035 raft_consensus.cc:374] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.535302 18035 raft_consensus.cc:729] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.535370 18035 consensus_queue.cc:260] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.535478 18035 ts_tablet_manager.cc:1428] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:48.535632 18041 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.535709 18041 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: This TxnStatusTablet replica's current role is: FOLLOWER
W20260927 18:43:48.560102 18013 tablet.cc:2369] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:43:48.562093 18042 raft_consensus.cc:482] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:48.562196 18042 raft_consensus.cc:504] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.562391 18042 leader_election.cc:288] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847)
I20260927 18:43:48.562573 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "391553716723429088b5e2cb66a5e5bf" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" is_pre_election: true
I20260927 18:43:48.562696 17686 raft_consensus.cc:2454] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:48.562654 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "391553716723429088b5e2cb66a5e5bf" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736" is_pre_election: true
I20260927 18:43:48.562736 17826 raft_consensus.cc:2454] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:48.562902 17899 leader_election.cc:302] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fe2952158741158f89045b1a1b5736, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:48.563009 18042 raft_consensus.cc:2781] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:48.563062 18042 raft_consensus.cc:482] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:48.563089 18042 raft_consensus.cc:3037] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.563908 18042 raft_consensus.cc:504] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.564073 18042 leader_election.cc:288] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Requested vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847)
I20260927 18:43:48.564266 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "391553716723429088b5e2cb66a5e5bf" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
I20260927 18:43:48.564273 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "391553716723429088b5e2cb66a5e5bf" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736"
I20260927 18:43:48.564352 17826 raft_consensus.cc:3037] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.564352 17686 raft_consensus.cc:3037] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.565121 17826 raft_consensus.cc:2454] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:48.565141 17686 raft_consensus.cc:2454] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:48.565318 17898 leader_election.cc:302] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4144cb3cc5ab410f904f6e1af23a83f7, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:48.565423 18042 raft_consensus.cc:2781] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:48.565474 18042 raft_consensus.cc:686] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [term 1 LEADER]: Becoming Leader. State: Replica: efd44071853246a4aa9410a8d822e917, State: Running, Role: LEADER
I20260927 18:43:48.565536 18042 consensus_queue.cc:237] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.565845 18045 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: TxnStatusTablet state changed. Reason: New leader efd44071853246a4aa9410a8d822e917. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.565932 18045 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:48.566208 17527 catalog_manager.cc:5495] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd44071853246a4aa9410a8d822e917 (127.15.110.195). New cstate: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } health_report { overall_health: HEALTHY } } }
I20260927 18:43:48.566465 18057 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:43:48.566586 18057 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:43:48.572470 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.569991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.15.110.195:35783)
0927 18:43:48.570036 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.570038 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.570047 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:48.570142 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.570142 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.570159 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.570217 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.570340 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.570811 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.571649 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.571861 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.571939 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.572134 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.572238 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.572329 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.572380 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.572401 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:48.572495 18052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.569920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.0.0.1:52446)
0927 18:43:48.569994 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.570006 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.570058 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.570219 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.570220 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.570221 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:48.570278 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.570280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.570909 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.570911 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.571549 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.571551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.571938 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.571939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.572008 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.572023 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.572308 (+ 285us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.572319 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.572375 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.572422 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:43:48.574281 17686 raft_consensus.cc:1261] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.574285 17826 raft_consensus.cc:1261] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.574621 18042 consensus_queue.cc:1035] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.574730 18045 consensus_queue.cc:1035] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.575959 18041 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: TxnStatusTablet state changed. Reason: New leader efd44071853246a4aa9410a8d822e917. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.575985 18043 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: TxnStatusTablet state changed. Reason: New leader efd44071853246a4aa9410a8d822e917. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.576071 18041 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.576071 18043 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.577215 18042 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.577306 18042 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:48.577268 18043 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.577337 18043 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P 4144cb3cc5ab410f904f6e1af23a83f7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.577350 18042 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.577394 18042 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P efd44071853246a4aa9410a8d822e917: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:43:48.577481 18060 mvcc.cc:205] Tried to move back new op lower bound from 7334029838637682688 to 7334029838604820480. Current Snapshot: MvccSnapshot[applied={T|T < 7334029838637682688}]
I20260927 18:43:48.577530 18041 tablet_replica.cc:439] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } } }
I20260927 18:43:48.577620 18041 tablet_replica.cc:442] T 391553716723429088b5e2cb66a5e5bf P a4fe2952158741158f89045b1a1b5736: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:43:48.578364 18061 mvcc.cc:205] Tried to move back new op lower bound from 7334029838637682688 to 7334029838604820480. Current Snapshot: MvccSnapshot[applied={T|T < 7334029838637682688}]
I20260927 18:43:48.582008 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.579747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.15.110.195:35783)
0927 18:43:48.579791 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.579793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.579802 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:48.579869 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.579870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.579882 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.579931 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.580068 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.580546 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.581309 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.581527 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.581604 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.581774 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.581807 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.581854 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.581910 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.581930 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:48.586907 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.584343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.15.110.195:45109)
0927 18:43:48.584417 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.584428 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.584473 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.584599 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.584600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.584601 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:48.584653 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.584654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.585475 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.585476 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.586080 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.586082 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.586530 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.586531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.586577 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.586590 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.586768 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.586770 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.586804 (+ 34us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.586837 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:43:48.586918 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.584407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:45109 (local address 127.15.110.195:35783)
0927 18:43:48.584464 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.584465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.584475 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:48.584514 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.584514 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.584524 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.584593 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.584712 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.585405 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.586165 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.586461 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.586532 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.586680 (+ 148us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.586711 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.586763 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.586807 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.586823 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:43:48.589093 17686 raft_consensus.cc:1261] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.589109 17826 raft_consensus.cc:1261] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.589325 18071 consensus_queue.cc:1035] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.589442 18042 consensus_queue.cc:1035] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.591310 18062 mvcc.cc:205] Tried to move back new op lower bound from 7334029838699761664 to 7334029838262665216. Current Snapshot: MvccSnapshot[applied={T|T < 7334029838699761664}]
I20260927 18:43:48.603592 17527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:48.603709 17527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:48.604497 17527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Sending DeleteTablet for 3 replicas of tablet 90535227e7bd4b75b03c65ac4110f76d
I20260927 18:43:48.604791 17945 tablet_service.cc:1514] Processing DeleteTablet for tablet 90535227e7bd4b75b03c65ac4110f76d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:48 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:52434
I20260927 18:43:48.604825 17666 tablet_service.cc:1514] Processing DeleteTablet for tablet 90535227e7bd4b75b03c65ac4110f76d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:48 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260927 18:43:48.604879 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet 90535227e7bd4b75b03c65ac4110f76d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:48 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:33012
I20260927 18:43:48.605137 18078 tablet_replica.cc:330] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: stopping tablet replica
I20260927 18:43:48.605175 18076 tablet_replica.cc:330] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: stopping tablet replica
I20260927 18:43:48.605304 18078 raft_consensus.cc:2229] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:48.605320 18076 raft_consensus.cc:2229] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:48.605377 18078 raft_consensus.cc:2258] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:48.605327 17527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:48.605458 17527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:48.605530 18077 tablet_replica.cc:330] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: stopping tablet replica
I20260927 18:43:48.605787 18076 raft_consensus.cc:2258] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:48.606074 18078 ts_tablet_manager.cc:1905] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:48.606287 18076 ts_tablet_manager.cc:1905] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:48.606666 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 (DEFAULT_TABLE table=test-table [id=13a582e1022748fbaefa76428b9969e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.606786 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e84b8de3a594b76a05fa2375d5da4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.606848 17945 tablet_service.cc:1467] Processing CreateTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 (DEFAULT_TABLE table=test-table [id=13a582e1022748fbaefa76428b9969e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.606942 17945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e84b8de3a594b76a05fa2375d5da4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.608144 18076 ts_tablet_manager.cc:1918] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:48.608209 18076 log.cc:1196] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/90535227e7bd4b75b03c65ac4110f76d
I20260927 18:43:48.608210 18037 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Bootstrap starting.
I20260927 18:43:48.608474 18036 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Bootstrap starting.
I20260927 18:43:48.608495 18076 ts_tablet_manager.cc:1939] T 90535227e7bd4b75b03c65ac4110f76d P efd44071853246a4aa9410a8d822e917: Deleting consensus metadata
I20260927 18:43:48.608613 18037 tablet_bootstrap.cc:654] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.608884 18036 tablet_bootstrap.cc:654] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.609349 17516 catalog_manager.cc:4841] TS efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783): tablet 90535227e7bd4b75b03c65ac4110f76d (table test-table [id=2815ae394e414a038f293d786d92d74c]) successfully deleted
I20260927 18:43:48.606666 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 (DEFAULT_TABLE table=test-table [id=13a582e1022748fbaefa76428b9969e2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:48.609467 18037 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: No bootstrap required, opened a new log
I20260927 18:43:48.609512 18037 ts_tablet_manager.cc:1397] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:48.609550 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e84b8de3a594b76a05fa2375d5da4e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:48.609655 18037 raft_consensus.cc:348] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.609730 18037 raft_consensus.cc:374] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.609751 18037 raft_consensus.cc:729] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.609797 18037 consensus_queue.cc:260] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.609889 18037 ts_tablet_manager.cc:1428] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:48.610585 18035 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Bootstrap starting.
I20260927 18:43:48.610982 18035 tablet_bootstrap.cc:654] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:48.611687 18035 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: No bootstrap required, opened a new log
I20260927 18:43:48.611742 18035 ts_tablet_manager.cc:1397] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:48.611902 18035 raft_consensus.cc:348] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.611976 18035 raft_consensus.cc:374] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.611999 18035 raft_consensus.cc:729] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.612043 18035 consensus_queue.cc:260] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.612133 18035 ts_tablet_manager.cc:1428] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:48.612860 18078 ts_tablet_manager.cc:1918] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:48.612946 18078 log.cc:1196] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/90535227e7bd4b75b03c65ac4110f76d
I20260927 18:43:48.613200 18078 ts_tablet_manager.cc:1939] T 90535227e7bd4b75b03c65ac4110f76d P a4fe2952158741158f89045b1a1b5736: Deleting consensus metadata
I20260927 18:43:48.613621 18077 raft_consensus.cc:2229] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:48.613607 17515 catalog_manager.cc:4841] TS a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847): tablet 90535227e7bd4b75b03c65ac4110f76d (table test-table [id=2815ae394e414a038f293d786d92d74c]) successfully deleted
I20260927 18:43:48.613705 18077 raft_consensus.cc:2258] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:48.614141 18036 tablet_bootstrap.cc:492] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: No bootstrap required, opened a new log
I20260927 18:43:48.614209 18036 ts_tablet_manager.cc:1397] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260927 18:43:48.614357 18036 raft_consensus.cc:348] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.614432 18036 raft_consensus.cc:374] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:48.614464 18036 raft_consensus.cc:729] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd44071853246a4aa9410a8d822e917, State: Initialized, Role: FOLLOWER
I20260927 18:43:48.614513 18036 consensus_queue.cc:260] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.614599 18036 ts_tablet_manager.cc:1428] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:48.614848 18077 ts_tablet_manager.cc:1905] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:48.616323 18077 ts_tablet_manager.cc:1918] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:48.616405 18077 log.cc:1196] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/90535227e7bd4b75b03c65ac4110f76d
I20260927 18:43:48.616648 18077 ts_tablet_manager.cc:1939] T 90535227e7bd4b75b03c65ac4110f76d P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting consensus metadata
I20260927 18:43:48.617049 17514 catalog_manager.cc:4841] TS 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455): tablet 90535227e7bd4b75b03c65ac4110f76d (table test-table [id=2815ae394e414a038f293d786d92d74c]) successfully deleted
I20260927 18:43:48.798725 18043 raft_consensus.cc:482] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:48.798827 18043 raft_consensus.cc:504] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.799235 18043 leader_election.cc:288] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847), efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783)
I20260927 18:43:48.803025 18083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.799414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:43847 (local address 127.15.110.193:39189)
0927 18:43:48.799576 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.799586 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.799656 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.799825 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.799826 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.799828 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.799931 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.799934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.800808 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.800809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.802055 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.802058 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.802672 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.802779 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.802785 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.802870 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.802924 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260927 18:43:48.803048 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.799565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39189 (local address 127.15.110.194:43847)
0927 18:43:48.799666 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.799668 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.799677 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:48.799721 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.799722 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.799731 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.799781 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.800010 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.800717 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802194 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.802562 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802671 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.802681 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.802905 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.802961 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260927 18:43:48.803087 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.799522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:44327 (local address 127.15.110.195:35783)
0927 18:43:48.799622 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.799625 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.799636 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:43:48.799756 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.799757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.799768 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.799797 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:48.800040 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.800506 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802153 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.802714 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802794 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.802804 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.802967 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.803013 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260927 18:43:48.803323 17825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e84b8de3a594b76a05fa2375d5da4e5" candidate_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736" is_pre_election: true
I20260927 18:43:48.803445 17825 raft_consensus.cc:2454] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4144cb3cc5ab410f904f6e1af23a83f7 in term 0.
I20260927 18:43:48.803638 17619 leader_election.cc:302] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4144cb3cc5ab410f904f6e1af23a83f7, a4fe2952158741158f89045b1a1b5736; no voters:
I20260927 18:43:48.803774 18043 raft_consensus.cc:2781] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:48.803826 18043 raft_consensus.cc:482] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:48.803843 18082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.799406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.15.110.193:44327)
0927 18:43:48.799571 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.799585 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.799656 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.799890 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.799891 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.799892 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:48.799948 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.799949 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.800673 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.800674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.802054 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.802058 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.802806 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.802807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.802882 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.802885 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.803675 (+ 790us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.803727 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":78,"thread_start_us":72,"threads_started":1}
I20260927 18:43:48.803856 18043 raft_consensus.cc:3037] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.804098 17966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e84b8de3a594b76a05fa2375d5da4e5" candidate_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd44071853246a4aa9410a8d822e917" is_pre_election: true
I20260927 18:43:48.804237 17966 raft_consensus.cc:2454] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4144cb3cc5ab410f904f6e1af23a83f7 in term 0.
I20260927 18:43:48.804729 18043 raft_consensus.cc:504] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.804883 18043 leader_election.cc:288] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [CANDIDATE]: Term 1 election: Requested vote from peers a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847), efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783)
I20260927 18:43:48.805047 17825 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e84b8de3a594b76a05fa2375d5da4e5" candidate_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736"
I20260927 18:43:48.805055 17966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e84b8de3a594b76a05fa2375d5da4e5" candidate_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd44071853246a4aa9410a8d822e917"
I20260927 18:43:48.805124 17966 raft_consensus.cc:3037] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.805145 17825 raft_consensus.cc:3037] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:48.805776 17825 raft_consensus.cc:2454] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4144cb3cc5ab410f904f6e1af23a83f7 in term 1.
I20260927 18:43:48.805860 17966 raft_consensus.cc:2454] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4144cb3cc5ab410f904f6e1af23a83f7 in term 1.
I20260927 18:43:48.806021 17620 leader_election.cc:302] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4144cb3cc5ab410f904f6e1af23a83f7, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:48.806144 18043 raft_consensus.cc:2781] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:48.806319 18043 raft_consensus.cc:686] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 LEADER]: Becoming Leader. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Running, Role: LEADER
I20260927 18:43:48.806398 18043 consensus_queue.cc:237] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:48.807302 17527 catalog_manager.cc:5495] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193). New cstate: current_term: 1 leader_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:48.840639 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.838561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55172 (local address 127.15.110.193:39455)
0927 18:43:48.838607 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.838609 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.838619 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:43:48.838676 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.838677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.838695 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.838754 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.838836 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.839330 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.840006 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.840186 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.840236 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.840364 (+ 128us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.840470 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.840525 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.840558 (+ 33us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.840576 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:43:48.845530 18034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.843381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:54679 (local address 127.15.110.195:35783)
0927 18:43:48.843444 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.843446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.843585 (+ 139us) server_negotiation.cc:407] Connection header received
0927 18:43:48.843631 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.843631 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.843640 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.843680 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.843799 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.844249 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.844980 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.845153 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.845201 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.845346 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.845371 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.845407 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.845460 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.845473 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:43:48.845573 18088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.843307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.15.110.193:54679)
0927 18:43:48.843504 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.843515 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.843584 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.843689 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.843689 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.843691 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:48.843751 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.843752 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.844338 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.844339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.844911 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.844913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.845220 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.845221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.845267 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.845280 (+ 13us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.845423 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.845424 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.845458 (+ 34us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.845490 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":112,"threads_started":1}
I20260927 18:43:48.847817 17966 raft_consensus.cc:1261] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Refusing update from remote peer 4144cb3cc5ab410f904f6e1af23a83f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.847810 17825 raft_consensus.cc:1261] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Refusing update from remote peer 4144cb3cc5ab410f904f6e1af23a83f7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:48.848119 18043 consensus_queue.cc:1035] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.848294 18084 consensus_queue.cc:1035] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:48.855957 18033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.853413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:41061 (local address 127.15.110.193:39455)
0927 18:43:48.853454 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:48.853456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:48.853465 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:48.853489 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:48.853489 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:48.853501 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:48.853575 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:48.853746 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.854406 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.855176 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.855511 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.855624 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.855731 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.855765 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.855814 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:48.855878 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 18:43:48.855899 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:48.855965 18018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:48.853337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39455 (local address 127.15.110.195:41061)
0927 18:43:48.853375 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:48.853386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:48.853423 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:48.853620 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:48.853621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:48.853622 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:48.853669 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.853671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.854480 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.854481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.855045 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:48.855047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:48.855590 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:48.855592 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:48.855654 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:48.855659 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:48.855827 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:48.855828 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:43:48.855874 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:43:48.855912 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18}
W20260927 18:43:48.856312 18062 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:43:50.375716 17527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:50.375833 17527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:50.376791 17527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Sending DeleteTablet for 3 replicas of tablet 8e84b8de3a594b76a05fa2375d5da4e5
I20260927 18:43:50.377074 17666 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260927 18:43:50.377102 17945 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:52434
I20260927 18:43:50.377322 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e84b8de3a594b76a05fa2375d5da4e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:33012
I20260927 18:43:50.377485 18116 tablet_replica.cc:330] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: stopping tablet replica
I20260927 18:43:50.377532 18116 raft_consensus.cc:2229] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:50.377558 18116 raft_consensus.cc:2258] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.377782 18116 ts_tablet_manager.cc:1905] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.379343 18116 ts_tablet_manager.cc:1918] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:43:50.379438 18116 log.cc:1196] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/8e84b8de3a594b76a05fa2375d5da4e5
I20260927 18:43:50.379655 18114 tablet_replica.cc:330] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: stopping tablet replica
I20260927 18:43:50.379735 18116 ts_tablet_manager.cc:1939] T 8e84b8de3a594b76a05fa2375d5da4e5 P a4fe2952158741158f89045b1a1b5736: Deleting consensus metadata
I20260927 18:43:50.379753 18114 raft_consensus.cc:2229] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:50.379869 18114 raft_consensus.cc:2258] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.380045 17515 catalog_manager.cc:4841] TS a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847): tablet 8e84b8de3a594b76a05fa2375d5da4e5 (table test-table [id=13a582e1022748fbaefa76428b9969e2]) successfully deleted
I20260927 18:43:50.380100 18114 ts_tablet_manager.cc:1905] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.381413 18115 tablet_replica.cc:330] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: stopping tablet replica
I20260927 18:43:50.381467 18114 ts_tablet_manager.cc:1918] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:43:50.381500 18115 raft_consensus.cc:2229] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:50.381541 18115 raft_consensus.cc:2258] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.381577 18114 log.cc:1196] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/8e84b8de3a594b76a05fa2375d5da4e5
I20260927 18:43:50.381817 18115 ts_tablet_manager.cc:1905] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.381843 18114 ts_tablet_manager.cc:1939] T 8e84b8de3a594b76a05fa2375d5da4e5 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting consensus metadata
I20260927 18:43:50.382179 17514 catalog_manager.cc:4841] TS 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455): tablet 8e84b8de3a594b76a05fa2375d5da4e5 (table test-table [id=13a582e1022748fbaefa76428b9969e2]) successfully deleted
I20260927 18:43:50.383647 18115 ts_tablet_manager.cc:1918] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:43:50.383699 18115 log.cc:1196] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/8e84b8de3a594b76a05fa2375d5da4e5
I20260927 18:43:50.383919 18115 ts_tablet_manager.cc:1939] T 8e84b8de3a594b76a05fa2375d5da4e5 P efd44071853246a4aa9410a8d822e917: Deleting consensus metadata
I20260927 18:43:50.384482 17516 catalog_manager.cc:4841] TS efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783): tablet 8e84b8de3a594b76a05fa2375d5da4e5 (table test-table [id=13a582e1022748fbaefa76428b9969e2]) successfully deleted
Sep 27 18:43:50 dist-test-slave-vs05 krb5kdc[17484](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:43:50 dist-test-slave-vs05 krb5kdc[17484](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534630, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:50.405900 18123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.393483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50078 (local address 127.15.110.254:40485)
0927 18:43:50.393628 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.393630 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.393692 (+ 62us) server_negotiation.cc:407] Connection header received
0927 18:43:50.393754 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.393754 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.393764 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.393840 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:50.394401 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.394978 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.395926 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.396139 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.396213 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.403571 (+ 7358us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:50.403573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:50.403573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:50.403580 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:50.404952 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:50.405277 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:50.405278 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:50.405278 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:50.405332 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:50.405475 (+ 143us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:50.405476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:50.405476 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:50.405578 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:50.405661 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.405748 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.405814 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":94,"threads_started":1}
I20260927 18:43:50.407496 17527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:50.407644 17527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:50.409112 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 10d09159b43d4796958d10a42918a9b9 (DEFAULT_TABLE table=test-table [id=df7cbdfabe5f496998197490b35bc7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.409137 17945 tablet_service.cc:1467] Processing CreateTablet for tablet 10d09159b43d4796958d10a42918a9b9 (DEFAULT_TABLE table=test-table [id=df7cbdfabe5f496998197490b35bc7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.409272 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d09159b43d4796958d10a42918a9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.409272 17945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d09159b43d4796958d10a42918a9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.410394 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 10d09159b43d4796958d10a42918a9b9 (DEFAULT_TABLE table=test-table [id=df7cbdfabe5f496998197490b35bc7c9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.410523 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10d09159b43d4796958d10a42918a9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.410557 18126 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Bootstrap starting.
I20260927 18:43:50.410883 18126 tablet_bootstrap.cc:654] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.411550 18126 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: No bootstrap required, opened a new log
I20260927 18:43:50.411584 18126 ts_tablet_manager.cc:1397] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:50.411684 18126 raft_consensus.cc:348] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.411733 18126 raft_consensus.cc:374] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.411767 18126 raft_consensus.cc:729] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd44071853246a4aa9410a8d822e917, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.411758 18128 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Bootstrap starting.
I20260927 18:43:50.411840 18126 consensus_queue.cc:260] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.411988 18125 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Bootstrap starting.
I20260927 18:43:50.412074 18126 ts_tablet_manager.cc:1428] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:50.412246 18128 tablet_bootstrap.cc:654] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.412402 18125 tablet_bootstrap.cc:654] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.412983 18128 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: No bootstrap required, opened a new log
I20260927 18:43:50.413028 18125 tablet_bootstrap.cc:492] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: No bootstrap required, opened a new log
I20260927 18:43:50.413054 18128 ts_tablet_manager.cc:1397] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:50.413082 18125 ts_tablet_manager.cc:1397] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:50.413199 18125 raft_consensus.cc:348] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.413206 18128 raft_consensus.cc:348] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.413259 18125 raft_consensus.cc:374] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.413275 18125 raft_consensus.cc:729] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.413275 18128 raft_consensus.cc:374] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.413300 18128 raft_consensus.cc:729] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.413345 18125 consensus_queue.cc:260] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.413355 18128 consensus_queue.cc:260] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.413514 18128 ts_tablet_manager.cc:1428] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:50.413528 18125 ts_tablet_manager.cc:1428] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:50.705425 18131 raft_consensus.cc:482] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:50.705544 18131 raft_consensus.cc:504] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.705838 18131 leader_election.cc:288] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783)
I20260927 18:43:50.709009 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.706130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:45365 (local address 127.15.110.195:35783)
0927 18:43:50.706330 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.706332 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.706341 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:50.706370 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.706371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.706384 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.706513 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:50.706687 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.707262 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708238 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.708623 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708708 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.708716 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.708885 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.708927 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260927 18:43:50.709036 18133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.706049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.15.110.194:45365)
0927 18:43:50.706224 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:50.706236 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:50.706296 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:50.706519 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:50.706519 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:50.706522 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:50.706609 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.706611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.707344 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.707345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.708144 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.708146 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.708732 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.708801 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.708805 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:50.708868 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:43:50.708917 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":105,"threads_started":1}
I20260927 18:43:50.709059 18132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.706049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39455 (local address 127.15.110.194:47819)
0927 18:43:50.706349 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:50.706367 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:50.706504 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:50.706672 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:50.706672 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:50.706674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:43:50.706750 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.706752 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.707459 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.707460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.708192 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.708194 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.708766 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.708842 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.708846 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:43:50.708907 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:43:50.708953 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":3,"thread_start_us":103,"threads_started":1}
I20260927 18:43:50.709087 18135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.706157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:47819 (local address 127.15.110.193:39455)
0927 18:43:50.706310 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.706312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.706488 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:43:50.706573 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.706574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.706584 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.706670 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:43:50.706842 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.707352 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708286 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.708674 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.708758 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.708766 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.708932 (+ 166us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.708974 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":82,"threads_started":1}
I20260927 18:43:50.709385 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d09159b43d4796958d10a42918a9b9" candidate_uuid: "a4fe2952158741158f89045b1a1b5736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" is_pre_election: true
I20260927 18:43:50.709487 17686 raft_consensus.cc:2454] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fe2952158741158f89045b1a1b5736 in term 0.
I20260927 18:43:50.709487 17966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d09159b43d4796958d10a42918a9b9" candidate_uuid: "a4fe2952158741158f89045b1a1b5736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd44071853246a4aa9410a8d822e917" is_pre_election: true
I20260927 18:43:50.709580 17966 raft_consensus.cc:2454] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4fe2952158741158f89045b1a1b5736 in term 0.
I20260927 18:43:50.709681 17758 leader_election.cc:302] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4144cb3cc5ab410f904f6e1af23a83f7, a4fe2952158741158f89045b1a1b5736; no voters:
I20260927 18:43:50.709862 18131 raft_consensus.cc:2781] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:50.709900 18131 raft_consensus.cc:482] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:50.709941 18131 raft_consensus.cc:3037] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.710695 18131 raft_consensus.cc:504] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.710853 18131 leader_election.cc:288] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [CANDIDATE]: Term 1 election: Requested vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783)
I20260927 18:43:50.711011 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d09159b43d4796958d10a42918a9b9" candidate_uuid: "a4fe2952158741158f89045b1a1b5736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
I20260927 18:43:50.711027 17966 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10d09159b43d4796958d10a42918a9b9" candidate_uuid: "a4fe2952158741158f89045b1a1b5736" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "efd44071853246a4aa9410a8d822e917"
I20260927 18:43:50.711095 17966 raft_consensus.cc:3037] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.711095 17686 raft_consensus.cc:3037] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.711915 17966 raft_consensus.cc:2454] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fe2952158741158f89045b1a1b5736 in term 1.
I20260927 18:43:50.711908 17686 raft_consensus.cc:2454] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4fe2952158741158f89045b1a1b5736 in term 1.
I20260927 18:43:50.712090 17760 leader_election.cc:302] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fe2952158741158f89045b1a1b5736, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:50.712188 18131 raft_consensus.cc:2781] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:50.712318 18131 raft_consensus.cc:686] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 1 LEADER]: Becoming Leader. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Running, Role: LEADER
I20260927 18:43:50.712486 18131 consensus_queue.cc:237] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.713261 17527 catalog_manager.cc:5495] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4fe2952158741158f89045b1a1b5736 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "a4fe2952158741158f89045b1a1b5736" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } health_report { overall_health: UNKNOWN } } }
I20260927 18:43:50.768420 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.765629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.15.110.194:43847)
0927 18:43:50.765791 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.765793 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.765800 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:43:50.765832 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.765832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.765850 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.765942 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:50.766101 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.766665 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.767518 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.767799 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.767876 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.768052 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.768171 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.768258 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.768281 (+ 23us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.768324 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":81,"threads_started":1}
I20260927 18:43:50.773195 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.771067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:39141 (local address 127.15.110.195:35783)
0927 18:43:50.771104 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.771106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.771246 (+ 140us) server_negotiation.cc:407] Connection header received
0927 18:43:50.771277 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.771278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.771287 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.771329 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:50.771425 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.771857 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.772583 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.772780 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.772849 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.772998 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.773030 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.773067 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.773119 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.773139 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:43:50.773218 18142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.770907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:35783 (local address 127.15.110.194:39141)
0927 18:43:50.771155 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:50.771164 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:50.771234 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:50.771337 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:50.771338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:50.771339 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:50.771387 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.771388 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.771915 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.771916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.772485 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.772486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.772857 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.772858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.772910 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.772922 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.773079 (+ 157us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.773080 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:43:50.773111 (+ 31us) client_negotiation.cc:240] Negotiation successful
0927 18:43:50.773147 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":80,"threads_started":1}
I20260927 18:43:50.775271 17686 raft_consensus.cc:1261] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Refusing update from remote peer a4fe2952158741158f89045b1a1b5736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:50.775290 17966 raft_consensus.cc:1261] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Refusing update from remote peer a4fe2952158741158f89045b1a1b5736: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:50.775545 18131 consensus_queue.cc:1035] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:50.775709 18136 consensus_queue.cc:1035] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [LEADER]: Connected to new peer: Peer: permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:50.783773 18139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.780222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:48747 (local address 127.15.110.194:43847)
0927 18:43:50.780256 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.780259 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.781496 (+ 1237us) server_negotiation.cc:407] Connection header received
0927 18:43:50.781561 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.781562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.781573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.781627 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:50.781726 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.782197 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.783003 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.783279 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.783367 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.783524 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.783560 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.783633 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.783686 (+ 53us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.783708 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:50.783814 18147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.780172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:43847 (local address 127.15.110.195:48747)
0927 18:43:50.781413 (+ 1241us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:50.781423 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:50.781508 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:50.781630 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:50.781631 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:50.781632 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:43:50.781680 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.781681 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.782325 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.782325 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.782902 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:50.782904 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.783368 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.783369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:50.783426 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.783431 (+ 5us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.783637 (+ 206us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.783639 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:43:50.783686 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:43:50.783733 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":79,"threads_started":1}
I20260927 18:43:50.792364 17527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:50.792477 17527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:50.793190 17527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Sending DeleteTablet for 3 replicas of tablet 10d09159b43d4796958d10a42918a9b9
I20260927 18:43:50.793457 17945 tablet_service.cc:1514] Processing DeleteTablet for tablet 10d09159b43d4796958d10a42918a9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:52434
I20260927 18:43:50.793551 18115 tablet_replica.cc:330] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: stopping tablet replica
I20260927 18:43:50.793529 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet 10d09159b43d4796958d10a42918a9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:33012
I20260927 18:43:50.793622 18115 raft_consensus.cc:2229] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:50.793638 18116 tablet_replica.cc:330] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: stopping tablet replica
I20260927 18:43:50.793664 18115 raft_consensus.cc:2258] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.793700 18116 raft_consensus.cc:2229] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:50.793776 18116 raft_consensus.cc:2258] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.793989 18115 ts_tablet_manager.cc:1905] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.794420 17527 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:43:50.794559 17527 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:43:50.795607 17666 tablet_service.cc:1514] Processing DeleteTablet for tablet 10d09159b43d4796958d10a42918a9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:50 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260927 18:43:50.795749 18114 tablet_replica.cc:330] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: stopping tablet replica
I20260927 18:43:50.795755 18115 ts_tablet_manager.cc:1918] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:50.795804 18115 log.cc:1196] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/10d09159b43d4796958d10a42918a9b9
I20260927 18:43:50.795804 18114 raft_consensus.cc:2229] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:50.795847 18114 raft_consensus.cc:2258] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:50.796059 18115 ts_tablet_manager.cc:1939] T 10d09159b43d4796958d10a42918a9b9 P efd44071853246a4aa9410a8d822e917: Deleting consensus metadata
I20260927 18:43:50.796188 18114 ts_tablet_manager.cc:1905] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.796381 17516 catalog_manager.cc:4841] TS efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783): tablet 10d09159b43d4796958d10a42918a9b9 (table test-table [id=df7cbdfabe5f496998197490b35bc7c9]) successfully deleted
I20260927 18:43:50.797396 18114 ts_tablet_manager.cc:1918] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:50.797415 18116 ts_tablet_manager.cc:1905] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:50.797447 18114 log.cc:1196] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/10d09159b43d4796958d10a42918a9b9
I20260927 18:43:50.797669 18114 ts_tablet_manager.cc:1939] T 10d09159b43d4796958d10a42918a9b9 P 4144cb3cc5ab410f904f6e1af23a83f7: Deleting consensus metadata
I20260927 18:43:50.797974 17514 catalog_manager.cc:4841] TS 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455): tablet 10d09159b43d4796958d10a42918a9b9 (table test-table [id=df7cbdfabe5f496998197490b35bc7c9]) successfully deleted
I20260927 18:43:50.798262 17666 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e4544253d41d39660e13466617291 (DEFAULT_TABLE table=test-table [id=b84a5bf99dd94c65a70283af40f29b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.798367 17666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e4544253d41d39660e13466617291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.798923 18116 ts_tablet_manager.cc:1918] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:43:50.798971 18116 log.cc:1196] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/10d09159b43d4796958d10a42918a9b9
I20260927 18:43:50.799211 18116 ts_tablet_manager.cc:1939] T 10d09159b43d4796958d10a42918a9b9 P a4fe2952158741158f89045b1a1b5736: Deleting consensus metadata
I20260927 18:43:50.799226 18125 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7: Bootstrap starting.
I20260927 18:43:50.799526 18125 tablet_bootstrap.cc:654] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.799717 17515 catalog_manager.cc:4841] TS a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847): tablet 10d09159b43d4796958d10a42918a9b9 (table test-table [id=df7cbdfabe5f496998197490b35bc7c9]) successfully deleted
I20260927 18:43:50.799747 17806 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e4544253d41d39660e13466617291 (DEFAULT_TABLE table=test-table [id=b84a5bf99dd94c65a70283af40f29b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.799852 17806 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e4544253d41d39660e13466617291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.799868 17945 tablet_service.cc:1467] Processing CreateTablet for tablet 8e8e4544253d41d39660e13466617291 (DEFAULT_TABLE table=test-table [id=b84a5bf99dd94c65a70283af40f29b60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:43:50.799979 17945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e8e4544253d41d39660e13466617291. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:50.800828 18126 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Bootstrap starting.
I20260927 18:43:50.800951 18128 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Bootstrap starting.
I20260927 18:43:50.801153 18126 tablet_bootstrap.cc:654] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.801339 18128 tablet_bootstrap.cc:654] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:50.801453 18125 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7: No bootstrap required, opened a new log
I20260927 18:43:50.801525 18125 ts_tablet_manager.cc:1397] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:43:50.801651 18125 raft_consensus.cc:348] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.801726 18125 raft_consensus.cc:374] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.801751 18125 raft_consensus.cc:729] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4144cb3cc5ab410f904f6e1af23a83f7, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.801777 18126 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: No bootstrap required, opened a new log
I20260927 18:43:50.801795 18125 consensus_queue.cc:260] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.801822 18126 ts_tablet_manager.cc:1397] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:50.801880 18125 ts_tablet_manager.cc:1428] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:50.801931 18126 raft_consensus.cc:348] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.801998 18126 raft_consensus.cc:374] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.802021 18126 raft_consensus.cc:729] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd44071853246a4aa9410a8d822e917, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.802042 18128 tablet_bootstrap.cc:492] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: No bootstrap required, opened a new log
I20260927 18:43:50.802062 18126 consensus_queue.cc:260] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.802080 18128 ts_tablet_manager.cc:1397] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:50.802181 18126 ts_tablet_manager.cc:1428] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:50.802253 18128 raft_consensus.cc:348] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.802338 18128 raft_consensus.cc:374] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:50.802367 18128 raft_consensus.cc:729] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fe2952158741158f89045b1a1b5736, State: Initialized, Role: FOLLOWER
I20260927 18:43:50.802430 18128 consensus_queue.cc:260] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.802528 18128 ts_tablet_manager.cc:1428] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:50.812665 18095 raft_consensus.cc:482] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:43:50.812737 18095 raft_consensus.cc:504] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.812884 18095 leader_election.cc:288] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847)
I20260927 18:43:50.813071 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e4544253d41d39660e13466617291" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" is_pre_election: true
I20260927 18:43:50.813119 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e4544253d41d39660e13466617291" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736" is_pre_election: true
I20260927 18:43:50.813166 17686 raft_consensus.cc:2454] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:50.813192 17826 raft_consensus.cc:2454] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 0.
I20260927 18:43:50.813342 17898 leader_election.cc:302] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4144cb3cc5ab410f904f6e1af23a83f7, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:50.813506 18095 raft_consensus.cc:2781] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:43:50.813539 18095 raft_consensus.cc:482] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:43:50.813552 18095 raft_consensus.cc:3037] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.814177 18095 raft_consensus.cc:504] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.814309 18095 leader_election.cc:288] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Requested vote from peers 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455), a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847)
I20260927 18:43:50.814429 17686 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e4544253d41d39660e13466617291" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4144cb3cc5ab410f904f6e1af23a83f7"
I20260927 18:43:50.814445 17826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e8e4544253d41d39660e13466617291" candidate_uuid: "efd44071853246a4aa9410a8d822e917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4fe2952158741158f89045b1a1b5736"
I20260927 18:43:50.814500 17686 raft_consensus.cc:3037] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.814500 17826 raft_consensus.cc:3037] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:50.815240 17826 raft_consensus.cc:2454] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:50.815239 17686 raft_consensus.cc:2454] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate efd44071853246a4aa9410a8d822e917 in term 1.
I20260927 18:43:50.815389 17899 leader_election.cc:302] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4fe2952158741158f89045b1a1b5736, efd44071853246a4aa9410a8d822e917; no voters:
I20260927 18:43:50.815486 18095 raft_consensus.cc:2781] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:50.815557 18095 raft_consensus.cc:686] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 1 LEADER]: Becoming Leader. State: Replica: efd44071853246a4aa9410a8d822e917, State: Running, Role: LEADER
I20260927 18:43:50.815608 18095 consensus_queue.cc:237] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } }
I20260927 18:43:50.816156 17528 catalog_manager.cc:5495] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 reported cstate change: term changed from 0 to 1, leader changed from <none> to efd44071853246a4aa9410a8d822e917 (127.15.110.195). New cstate: current_term: 1 leader_uuid: "efd44071853246a4aa9410a8d822e917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "efd44071853246a4aa9410a8d822e917" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 35783 } health_report { overall_health: HEALTHY } } }
I20260927 18:43:50.826858 18134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:50.824469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33994 (local address 127.15.110.195:35783)
0927 18:43:50.824541 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:50.824543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:50.824552 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:43:50.824576 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:50.824577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:50.824590 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:50.824652 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:43:50.824770 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.825325 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.826130 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:50.826337 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:50.826422 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:50.826604 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:43:50.826637 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:43:50.826688 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:50.826754 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:43:50.826776 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:43:50.829365 17826 raft_consensus.cc:1261] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:50.829418 17686 raft_consensus.cc:1261] T 8e8e4544253d41d39660e13466617291 P 4144cb3cc5ab410f904f6e1af23a83f7 [term 1 FOLLOWER]: Refusing update from remote peer efd44071853246a4aa9410a8d822e917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:43:50.829559 18100 consensus_queue.cc:1035] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4fe2952158741158f89045b1a1b5736" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 43847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:43:50.829646 18071 consensus_queue.cc:1035] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4144cb3cc5ab410f904f6e1af23a83f7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:43:50.834028 18093 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260927 18:43:52.351773 17527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:52.351883 17527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:43:52.352767 17527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e0d46c74c12a46a98f8496f9f5dbeeb8: Sending DeleteTablet for 3 replicas of tablet 8e8e4544253d41d39660e13466617291
I20260927 18:43:52.353036 17945 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e4544253d41d39660e13466617291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:52 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:52434
I20260927 18:43:52.353060 17806 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e4544253d41d39660e13466617291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:52 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:33012
I20260927 18:43:52.353173 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17601
I20260927 18:43:52.353233 17666 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e8e4544253d41d39660e13466617291 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:43:52 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:55162
I20260927 18:43:52.353761 18176 tablet_replica.cc:330] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: stopping tablet replica
I20260927 18:43:52.353825 18176 raft_consensus.cc:2229] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:43:52.353863 18176 raft_consensus.cc:2258] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:52.354053 18175 tablet_replica.cc:330] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: stopping tablet replica
I20260927 18:43:52.354138 18175 raft_consensus.cc:2229] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:43:52.354219 18176 ts_tablet_manager.cc:1905] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:52.354249 18175 raft_consensus.cc:2258] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:43:52.354468 18175 ts_tablet_manager.cc:1905] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:43:52.355702 18176 ts_tablet_manager.cc:1918] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:43:52.355777 18176 log.cc:1196] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/8e8e4544253d41d39660e13466617291
I20260927 18:43:52.356000 18176 ts_tablet_manager.cc:1939] T 8e8e4544253d41d39660e13466617291 P a4fe2952158741158f89045b1a1b5736: Deleting consensus metadata
I20260927 18:43:52.356310 17515 catalog_manager.cc:4841] TS a4fe2952158741158f89045b1a1b5736 (127.15.110.194:43847): tablet 8e8e4544253d41d39660e13466617291 (table test-table [id=b84a5bf99dd94c65a70283af40f29b60]) successfully deleted
I20260927 18:43:52.356462 18175 ts_tablet_manager.cc:1918] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260927 18:43:52.356550 18175 log.cc:1196] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/8e8e4544253d41d39660e13466617291
I20260927 18:43:52.356817 18175 ts_tablet_manager.cc:1939] T 8e8e4544253d41d39660e13466617291 P efd44071853246a4aa9410a8d822e917: Deleting consensus metadata
I20260927 18:43:52.357133 17516 catalog_manager.cc:4841] TS efd44071853246a4aa9410a8d822e917 (127.15.110.195:35783): tablet 8e8e4544253d41d39660e13466617291 (table test-table [id=b84a5bf99dd94c65a70283af40f29b60]) successfully deleted
W20260927 18:43:52.360955 17514 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:39455 (error 108)
I20260927 18:43:52.361186 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17741
I20260927 18:43:52.362001 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:52.361327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39455 (local address 127.0.0.1:49024)
0927 18:43:52.361873 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:52.361929 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":56,"threads_started":1}
W20260927 18:43:52.362147 17514 catalog_manager.cc:4584] TS 4144cb3cc5ab410f904f6e1af23a83f7 (127.15.110.193:39455): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e8e4544253d41d39660e13466617291: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39455: connect: Connection refused (error 111)
I20260927 18:43:52.367945 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17881
I20260927 18:43:52.373638 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 17497
2026-09-27T18:43:52Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5892 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:52 dist-test-slave-vs05 krb5kdc[18181](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:52 dist-test-slave-vs05 krb5kdc[18181](info): set up 2 sockets
Sep 27 18:43:52 dist-test-slave-vs05 krb5kdc[18181](info): commencing operation
krb5kdc: starting...
W20260927 18:43:53.410941 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:53.528422 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46619
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:53.608820 18194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:53.609026 18194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:53.609048 18194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:53.610129 18194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:53.610167 18194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:53.610181 18194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:53.610193 18194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:53.610205 18194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:53.611624 18194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:53.611815 18194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:53.611987 18194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:53.617118 18199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:53.617123 18202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:53.617238 18194 server_base.cc:1034] running on GCE node
W20260927 18:43:53.617156 18200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:53.617625 18194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:53.617885 18194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:53.619029 18194 hybrid_clock.cc:648] HybridClock initialized: now 1790534633619000 us; error 48 us; skew 500 ppm
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:53.621582 18194 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:53.622041 18194 webserver.cc:460] Webserver started at http://127.15.110.254:36511/ using document root <none> and password file <none>
I20260927 18:43:53.622294 18194 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:53.622341 18194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:53.622493 18194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:53.623688 18194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "77fa78ce13a2f14ac3fb186b8a6b3904"
server_key_iv: "f5a67fe48b340c0e9856f14e05e4321c"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.624050 18194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "77fa78ce13a2f14ac3fb186b8a6b3904"
server_key_iv: "f5a67fe48b340c0e9856f14e05e4321c"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.625356 18194 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:53.626111 18209 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.626317 18194 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:53.626401 18194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "77fa78ce13a2f14ac3fb186b8a6b3904"
server_key_iv: "f5a67fe48b340c0e9856f14e05e4321c"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.626459 18194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:53.640676 18194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:53.642891 18194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:53.643057 18194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:53.647073 18194 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46619
I20260927 18:43:53.647176 18261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46619 every 8 connection(s)
I20260927 18:43:53.647746 18194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:53.648725 18262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:53.651021 18262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3: Bootstrap starting.
I20260927 18:43:53.651633 18262 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:53.651923 18262 log.cc:824] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:53.652627 18262 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3: No bootstrap required, opened a new log
I20260927 18:43:53.653129 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18194
I20260927 18:43:53.653239 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:53.653399 15803 external_mini_cluster.cc:1428] Setting key 5dd052e43988db60e9d13241a041132e
I20260927 18:43:53.654744 18262 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } }
I20260927 18:43:53.654951 18262 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:53.655004 18262 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e50f1cc7b8a40e485bc3eefb9c045e3, State: Initialized, Role: FOLLOWER
I20260927 18:43:53.655170 18262 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } }
I20260927 18:43:53.655300 18262 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:53.655334 18262 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:53.655388 18262 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:53.656122 18262 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } }
I20260927 18:43:53.656248 18262 leader_election.cc:302] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e50f1cc7b8a40e485bc3eefb9c045e3; no voters:
I20260927 18:43:53.656459 18262 leader_election.cc:288] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:53.656491 18267 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:53.656657 18267 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [term 1 LEADER]: Becoming Leader. State: Replica: 9e50f1cc7b8a40e485bc3eefb9c045e3, State: Running, Role: LEADER
I20260927 18:43:53.656806 18267 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } }
I20260927 18:43:53.656795 18262 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:53.657289 18268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } } }
I20260927 18:43:53.657434 18268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:53.657187 18269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e50f1cc7b8a40e485bc3eefb9c045e3. Latest consensus state: current_term: 1 leader_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e50f1cc7b8a40e485bc3eefb9c045e3" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46619 } } }
I20260927 18:43:53.657681 18269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:53.657752 18271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:53.658262 18271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:53.660355 18271 catalog_manager.cc:1347] Generated new cluster ID: 2a7a2cb8943a46e3b8a341633d2078f6
I20260927 18:43:53.660418 18271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:53.665642 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:53.653917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.15.110.254:46619)
0927 18:43:53.654090 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:53.654098 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:53.654122 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:43:53.654872 (+ 750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:53.654878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:53.654926 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:53.655087 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:53.655980 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.657255 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.658358 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.658985 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.659117 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:53.661957 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:53.661961 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:53.661961 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:53.661979 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:53.663962 (+ 1983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.664380 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.664383 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.664383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.664481 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.664726 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.664727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.664727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.665017 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:53.665157 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:53.665290 (+ 133us) server_negotiation.cc:299] Negotiation successful
0927 18:43:53.665417 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260927 18:43:53.668025 18271 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:53.668499 18271 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:53.671845 18271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e50f1cc7b8a40e485bc3eefb9c045e3: Generated new TSK 0
I20260927 18:43:53.672046 18271 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:53.698450 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:53.778227 18288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:53.778425 18288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:53.778450 18288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:53.779646 18288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:53.779686 18288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:53.779713 18288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:53.781119 18288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:53.781349 18288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:53.781592 18288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:53.786499 18296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:53.786509 18293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:53.786509 18294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:53.786835 18288 server_base.cc:1034] running on GCE node
I20260927 18:43:53.786984 18288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:53.787216 18288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:53.788374 18288 hybrid_clock.cc:648] HybridClock initialized: now 1790534633788362 us; error 26 us; skew 500 ppm
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:53.790820 18288 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:53.791391 18288 webserver.cc:460] Webserver started at http://127.15.110.193:36987/ using document root <none> and password file <none>
I20260927 18:43:53.791751 18288 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:53.791829 18288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:53.792001 18288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:53.793211 18288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "eb5f65dc877c443895eeb74a2a18d57c"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "5081d5a1d45d8f86c5b108e3e1556a0e"
server_key_iv: "583f7bd5fedc5403a8648b2e07bd80c9"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.793613 18288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "eb5f65dc877c443895eeb74a2a18d57c"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "5081d5a1d45d8f86c5b108e3e1556a0e"
server_key_iv: "583f7bd5fedc5403a8648b2e07bd80c9"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.795078 18288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:53.796013 18303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.796211 18288 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:53.796309 18288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "eb5f65dc877c443895eeb74a2a18d57c"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "5081d5a1d45d8f86c5b108e3e1556a0e"
server_key_iv: "583f7bd5fedc5403a8648b2e07bd80c9"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.796396 18288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:53.812152 18288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:53.814335 18288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:53.814517 18288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:53.814770 18288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:53.815093 18288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:53.815155 18288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.815212 18288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:53.815269 18288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.821525 18288 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:44053
I20260927 18:43:53.821568 18416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:44053 every 8 connection(s)
I20260927 18:43:53.821871 18288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:53.823105 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18288
I20260927 18:43:53.823225 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:53.823393 15803 external_mini_cluster.cc:1428] Setting key 7aabff8bfe77a5acef9b22c9cb7f4024
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:43:53.832150 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:53.822571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56925 (local address 127.15.110.254:46619)
0927 18:43:53.822633 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:53.822635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:53.822813 (+ 178us) server_negotiation.cc:407] Connection header received
0927 18:43:53.823670 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:53.823671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:53.823681 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:53.823748 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:53.825007 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.825493 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.826497 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.826740 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.826817 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:53.829259 (+ 2442us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:53.829260 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:53.829261 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:53.829269 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:53.830705 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.831127 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.831129 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.831129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.831234 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.831494 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.831495 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.831495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.831617 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:53.831696 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:53.831964 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 18:43:53.832058 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:43:53.832376 18419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:53.822534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46619 (local address 127.15.110.193:56925)
0927 18:43:53.822689 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:53.822731 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:53.823471 (+ 740us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:53.823824 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:53.823828 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:53.824626 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:53.824901 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:53.824907 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.825595 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.825596 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:53.826404 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:53.826405 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.826837 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.826838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:53.826964 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:53.827517 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:53.827528 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:53.829127 (+ 1599us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:53.830808 (+ 1681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:53.830815 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:53.830817 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:53.831049 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:53.831311 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:53.831312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:53.831312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:53.831425 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:53.831686 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:53.831692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:53.831834 (+ 142us) client_negotiation.cc:769] Sending connection context
0927 18:43:53.831945 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:43:53.832102 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
I20260927 18:43:53.832871 18417 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46619
I20260927 18:43:53.832966 18417 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:53.833163 18417 heartbeater.cc:511] Master 127.15.110.254:46619 requested a full tablet report, sending...
I20260927 18:43:53.833807 18226 ts_manager.cc:194] Registered new tserver with Master: eb5f65dc877c443895eeb74a2a18d57c (127.15.110.193:44053)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
I20260927 18:43:53.834435 18226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:56925
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:53.850203 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:53.933045 18422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:53.933260 18422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:53.933284 18422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:53.934387 18422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:53.934458 18422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:53.934507 18422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:53.935952 18422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:53.936136 18422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:53.936311 18422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:53.941424 18428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:53.941527 18427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:53.941382 18430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:53.941635 18422 server_base.cc:1034] running on GCE node
I20260927 18:43:53.941936 18422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:53.942212 18422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:53.943382 18422 hybrid_clock.cc:648] HybridClock initialized: now 1790534633943368 us; error 47 us; skew 500 ppm
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:53.946126 18422 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:53.946759 18422 webserver.cc:460] Webserver started at http://127.15.110.194:32843/ using document root <none> and password file <none>
I20260927 18:43:53.947017 18422 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:53.947088 18422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:53.947278 18422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:53.948367 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "1466732ef9854fa891c59209000e0a65"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "738cf5ae3f446b65391cfdce6522e8d6"
server_key_iv: "0ab408808da45e19b42e85bf02f2b1fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.948755 18422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "1466732ef9854fa891c59209000e0a65"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "738cf5ae3f446b65391cfdce6522e8d6"
server_key_iv: "0ab408808da45e19b42e85bf02f2b1fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.950158 18422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:53.951032 18437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.951308 18422 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:53.951417 18422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "1466732ef9854fa891c59209000e0a65"
format_stamp: "Formatted at 2026-09-27 18:43:53 on dist-test-slave-vs05"
server_key: "738cf5ae3f446b65391cfdce6522e8d6"
server_key_iv: "0ab408808da45e19b42e85bf02f2b1fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:53.951547 18422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:53.963737 18422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:53.965938 18422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:53.966122 18422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:53.966357 18422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:53.966678 18422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:53.966734 18422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.966775 18422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:53.966802 18422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:53.972599 18422 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:42749
I20260927 18:43:53.972649 18550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:42749 every 8 connection(s)
I20260927 18:43:53.972959 18422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:53.975238 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18422
I20260927 18:43:53.975337 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:53.975539 15803 external_mini_cluster.cc:1428] Setting key 59a6df84156e414f1336d7e44f08c2fc
Sep 27 18:43:53 dist-test-slave-vs05 krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534633, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:53.983893 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:53.973653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:40113 (local address 127.15.110.254:46619)
0927 18:43:53.973695 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:53.973698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:53.973926 (+ 228us) server_negotiation.cc:407] Connection header received
0927 18:43:53.974384 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:53.974385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:53.974395 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:53.974456 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:53.975739 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.976374 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.977873 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.978210 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.978283 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:53.980634 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:53.980636 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:53.980637 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:53.980645 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:53.982324 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.982779 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.982781 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.982781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.982845 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:53.983205 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:53.983206 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:53.983207 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:53.983352 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:53.983429 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:53.983692 (+ 263us) server_negotiation.cc:299] Negotiation successful
0927 18:43:53.983774 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:53.983916 18553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:53.973626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46619 (local address 127.15.110.194:40113)
0927 18:43:53.973791 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:53.973832 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:53.974306 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:53.974567 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:53.974571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:53.975324 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:53.975626 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:53.975636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.976459 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.976461 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:53.977779 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:53.977783 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:53.978364 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:53.978365 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:53.978513 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:53.979055 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:53.979075 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:53.980533 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:53.982426 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:53.982432 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:53.982433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:53.982702 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:53.982954 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:53.982955 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:53.982956 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:53.983096 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:53.983424 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:53.983429 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:53.983550 (+ 121us) client_negotiation.cc:769] Sending connection context
0927 18:43:53.983636 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:43:53.983755 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260927 18:43:53.984393 18551 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46619
I20260927 18:43:53.984509 18551 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
I20260927 18:43:53.984726 18551 heartbeater.cc:511] Master 127.15.110.254:46619 requested a full tablet report, sending...
I20260927 18:43:53.985170 18226 ts_manager.cc:194] Registered new tserver with Master: 1466732ef9854fa891c59209000e0a65 (127.15.110.194:42749)
I20260927 18:43:53.985661 18226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:40113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:54.002979 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:54.084390 18556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:54.084600 18556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:54.084625 18556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:54.085744 18556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:54.085781 18556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:54.085806 18556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:54.087214 18556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:54.087496 18556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:54.087687 18556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:54.092555 18564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:54.092571 18562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:54.092553 18561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:54.092811 18556 server_base.cc:1034] running on GCE node
I20260927 18:43:54.093056 18556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:54.093307 18556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:54.094489 18556 hybrid_clock.cc:648] HybridClock initialized: now 1790534634094483 us; error 35 us; skew 500 ppm
Sep 27 18:43:54 dist-test-slave-vs05 krb5kdc[18181](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:54.097153 18556 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:54.097682 18556 webserver.cc:460] Webserver started at http://127.15.110.195:45877/ using document root <none> and password file <none>
I20260927 18:43:54.097920 18556 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:54.097996 18556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:54.098150 18556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:54.099305 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "e0ad793996e04259a5be8df3050c1ff6"
format_stamp: "Formatted at 2026-09-27 18:43:54 on dist-test-slave-vs05"
server_key: "a8cd022a7fe96799b469d6383ee277d2"
server_key_iv: "068a2825924fe68b74c6f01d472d910c"
server_key_version: "encryptionkey@0"
I20260927 18:43:54.099671 18556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "e0ad793996e04259a5be8df3050c1ff6"
format_stamp: "Formatted at 2026-09-27 18:43:54 on dist-test-slave-vs05"
server_key: "a8cd022a7fe96799b469d6383ee277d2"
server_key_iv: "068a2825924fe68b74c6f01d472d910c"
server_key_version: "encryptionkey@0"
I20260927 18:43:54.100994 18556 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:54.101752 18571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:54.101961 18556 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:54.102068 18556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "e0ad793996e04259a5be8df3050c1ff6"
format_stamp: "Formatted at 2026-09-27 18:43:54 on dist-test-slave-vs05"
server_key: "a8cd022a7fe96799b469d6383ee277d2"
server_key_iv: "068a2825924fe68b74c6f01d472d910c"
server_key_version: "encryptionkey@0"
I20260927 18:43:54.102156 18556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:54.113145 18556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:54.114928 18556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:54.115085 18556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:54.115337 18556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:54.115651 18556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:54.115705 18556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:54.115746 18556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:54.115773 18556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:54.121819 18556 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:35067
I20260927 18:43:54.121855 18684 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:35067 every 8 connection(s)
I20260927 18:43:54.122206 18556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:54.128086 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18556
I20260927 18:43:54.128197 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:54.128386 15803 external_mini_cluster.cc:1428] Setting key 82e7280055c34db39e43fc1214c85df8
Sep 27 18:43:54 dist-test-slave-vs05 krb5kdc[18181](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534634, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:54.131973 18266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:54.122994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:33153 (local address 127.15.110.254:46619)
0927 18:43:54.123034 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:54.123036 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:54.123334 (+ 298us) server_negotiation.cc:407] Connection header received
0927 18:43:54.123868 (+ 534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:54.123869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:54.123880 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:54.123953 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:54.124899 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:54.125406 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:54.126511 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:54.126708 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:54.126797 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:54.129328 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:54.129329 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:54.129330 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:54.129337 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:54.130548 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:54.130944 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:54.130945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:54.130945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:54.131027 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:54.131336 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:54.131337 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:54.131337 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:54.131455 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:54.131533 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:54.131799 (+ 266us) server_negotiation.cc:299] Negotiation successful
0927 18:43:54.131876 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:54.132166 18687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:54.122984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46619 (local address 127.15.110.195:33153)
0927 18:43:54.123153 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:54.123236 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:54.123758 (+ 522us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:54.124026 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:54.124030 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:54.124597 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:54.124785 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:54.124789 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:54.125508 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:54.125509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:54.126408 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:54.126411 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:54.126816 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:54.126817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:54.126936 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:54.127519 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:54.127527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:54.129249 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:54.130629 (+ 1380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:54.130635 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:54.130637 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:54.130852 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:54.131092 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:54.131093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:54.131094 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:54.131261 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:54.131522 (+ 261us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:54.131528 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:54.131666 (+ 138us) client_negotiation.cc:769] Sending connection context
0927 18:43:54.131768 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:43:54.131918 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":90,"threads_started":1}
I20260927 18:43:54.132668 18685 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46619
I20260927 18:43:54.132773 18685 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:54.132948 18685 heartbeater.cc:511] Master 127.15.110.254:46619 requested a full tablet report, sending...
I20260927 18:43:54.133342 18226 ts_manager.cc:194] Registered new tserver with Master: e0ad793996e04259a5be8df3050c1ff6 (127.15.110.195:35067)
I20260927 18:43:54.133706 18226 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:33153
I20260927 18:43:54.140452 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:43:54.151803 18266 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:43:54.150771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46634 (local address 127.15.110.254:46619)
0927 18:43:54.150849 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:54.150851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:54.150956 (+ 105us) server_negotiation.cc:407] Connection header received
0927 18:43:54.151063 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:54.151064 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:54.151074 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:54.151145 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:54.151748 (+ 603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46634: BlockingRecv error: recv got EOF from 127.0.0.1:46634 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:43:54.152850 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18288
I20260927 18:43:54.157459 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18422
I20260927 18:43:54.161952 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18556
I20260927 18:43:54.166356 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18194
2026-09-27T18:43:54Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1791 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:54 dist-test-slave-vs05 krb5kdc[18697](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:54 dist-test-slave-vs05 krb5kdc[18697](info): set up 2 sockets
Sep 27 18:43:54 dist-test-slave-vs05 krb5kdc[18697](info): commencing operation
krb5kdc: starting...
W20260927 18:43:55.203469 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:55.320603 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35443
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36225
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:35443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:55.402002 18710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:55.402257 18710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:55.402298 18710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:55.403513 18710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:55.403589 18710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:55.403614 18710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:55.403636 18710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:55.403666 18710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:55.405148 18710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36225
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:35443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:55.405422 18710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:55.405659 18710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:55.410517 18718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:55.410584 18716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:55.410534 18715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:55.410837 18710 server_base.cc:1034] running on GCE node
I20260927 18:43:55.411084 18710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:55.411392 18710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:55.412583 18710 hybrid_clock.cc:648] HybridClock initialized: now 1790534635412524 us; error 32 us; skew 500 ppm
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18697](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534635, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:55.415027 18710 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:55.415503 18710 webserver.cc:460] Webserver started at http://127.15.110.254:42659/ using document root <none> and password file <none>
I20260927 18:43:55.415743 18710 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:55.415817 18710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:55.416018 18710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:55.417176 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a"
format_stamp: "Formatted at 2026-09-27 18:43:55 on dist-test-slave-vs05"
server_key: "32a405d7d92d4143cc85ae3325eca230"
server_key_iv: "fd65050c4b9c4647a86d1782e90f0586"
server_key_version: "encryptionkey@0"
I20260927 18:43:55.417547 18710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a"
format_stamp: "Formatted at 2026-09-27 18:43:55 on dist-test-slave-vs05"
server_key: "32a405d7d92d4143cc85ae3325eca230"
server_key_iv: "fd65050c4b9c4647a86d1782e90f0586"
server_key_version: "encryptionkey@0"
I20260927 18:43:55.419039 18710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:55.419893 18725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:55.420105 18710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:55.420220 18710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a"
format_stamp: "Formatted at 2026-09-27 18:43:55 on dist-test-slave-vs05"
server_key: "32a405d7d92d4143cc85ae3325eca230"
server_key_iv: "fd65050c4b9c4647a86d1782e90f0586"
server_key_version: "encryptionkey@0"
I20260927 18:43:55.420305 18710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:55.449569 18710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:55.452016 18710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:55.452224 18710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:55.456472 18710 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:35443
I20260927 18:43:55.456532 18777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:35443 every 8 connection(s)
I20260927 18:43:55.457073 18710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:55.457785 18778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:55.460244 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a: Bootstrap starting.
I20260927 18:43:55.460886 18778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:55.461202 18778 log.cc:824] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:55.462002 18778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a: No bootstrap required, opened a new log
I20260927 18:43:55.463802 18778 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } }
I20260927 18:43:55.463953 18778 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:55.463977 18778 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffc06a2b26f744e6ae379ce7a90d6e0a, State: Initialized, Role: FOLLOWER
I20260927 18:43:55.464138 18778 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } }
I20260927 18:43:55.464214 18778 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:55.464236 18778 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:55.464306 18778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:55.465225 18778 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } }
I20260927 18:43:55.465339 18778 leader_election.cc:302] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffc06a2b26f744e6ae379ce7a90d6e0a; no voters:
I20260927 18:43:55.465538 18778 leader_election.cc:288] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:55.465620 18781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:55.465823 18781 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [term 1 LEADER]: Becoming Leader. State: Replica: ffc06a2b26f744e6ae379ce7a90d6e0a, State: Running, Role: LEADER
I20260927 18:43:55.465870 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18710
I20260927 18:43:55.465960 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:55.465821 18778 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:55.465963 18781 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } }
I20260927 18:43:55.466136 15803 external_mini_cluster.cc:1428] Setting key 188e2ffdf3076b69e6af84190fc6881a
I20260927 18:43:55.466370 18782 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } } }
I20260927 18:43:55.466449 18782 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:55.466658 18783 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffc06a2b26f744e6ae379ce7a90d6e0a. Latest consensus state: current_term: 1 leader_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffc06a2b26f744e6ae379ce7a90d6e0a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35443 } } }
I20260927 18:43:55.466713 18783 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:55.467064 18788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:55.467657 18788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:55.469362 18788 catalog_manager.cc:1347] Generated new cluster ID: 86078f3022914a16af0995309c228745
I20260927 18:43:55.469416 18788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18697](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534635, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:55.481340 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:55.466780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42618 (local address 127.15.110.254:35443)
0927 18:43:55.467070 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:55.467080 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:55.467275 (+ 195us) server_negotiation.cc:407] Connection header received
0927 18:43:55.467514 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:55.467522 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:55.467613 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:55.467763 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:55.468682 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:55.469868 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:55.470610 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:55.471095 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:55.471263 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:55.477820 (+ 6557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:55.477825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:55.477826 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:55.477848 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:55.479690 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:55.480089 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:55.480093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:55.480093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:55.480190 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:55.480400 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:55.480401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:55.480402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:55.480690 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:55.480863 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:55.480973 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 18:43:55.481117 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":101,"threads_started":1}
I20260927 18:43:55.482239 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:43:55.491513 18788 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:55.492244 18788 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:55.498472 18788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffc06a2b26f744e6ae379ce7a90d6e0a: Generated new TSK 0
I20260927 18:43:55.498662 18788 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:43:55.502250 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:55.498661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42634 (local address 127.15.110.254:35443)
0927 18:43:55.498704 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:55.498707 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:55.498888 (+ 181us) server_negotiation.cc:407] Connection header received
0927 18:43:55.498979 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:55.498979 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:55.498990 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:55.499037 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:55.499490 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:55.499994 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:55.500772 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:55.500991 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:55.501076 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:55.501787 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:55.501788 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:55.501788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:43:55.501976 (+ 188us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:55.501991 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:43:55.501997 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:55.502061 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:55.502131 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 18:43:55.502160 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:43:55.504818 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18710
2026-09-27T18:43:55Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1335 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18811](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18811](info): set up 2 sockets
Sep 27 18:43:55 dist-test-slave-vs05 krb5kdc[18811](info): commencing operation
krb5kdc: starting...
W20260927 18:43:56.551132 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.029s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:56 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:56.672495 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35259
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:35259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:56.754426 18824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:56.754695 18824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:56.754719 18824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:56.756011 18824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:56.756079 18824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:56.756094 18824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:56.756129 18824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:56.756141 18824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:56.757843 18824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:35259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:35259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:56.758123 18824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:56.758358 18824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:56.763265 18830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:56.763273 18832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:56.763302 18824 server_base.cc:1034] running on GCE node
W20260927 18:43:56.763280 18829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:56.763715 18824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:56.763963 18824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:56.765133 18824 hybrid_clock.cc:648] HybridClock initialized: now 1790534636765120 us; error 31 us; skew 500 ppm
Sep 27 18:43:56 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:56.767951 18824 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:56.768450 18824 webserver.cc:460] Webserver started at http://127.15.110.254:38645/ using document root <none> and password file <none>
I20260927 18:43:56.768695 18824 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:56.768743 18824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:56.768883 18824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:56.769974 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "f0cce5ae2eca48b4a6e530c5ac668388"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "646a9d672ccb25951f876a2c2bd80fab"
server_key_iv: "6f6ed2f3b14e0c1efb912f719979a955"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.770347 18824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "f0cce5ae2eca48b4a6e530c5ac668388"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "646a9d672ccb25951f876a2c2bd80fab"
server_key_iv: "6f6ed2f3b14e0c1efb912f719979a955"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.771850 18824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:56.772804 18839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:56.773115 18824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:56.773200 18824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "f0cce5ae2eca48b4a6e530c5ac668388"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "646a9d672ccb25951f876a2c2bd80fab"
server_key_iv: "6f6ed2f3b14e0c1efb912f719979a955"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.773300 18824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:56.791421 18824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:56.793869 18824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:56.794049 18824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:56.798230 18824 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:35259
I20260927 18:43:56.798274 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:35259 every 8 connection(s)
I20260927 18:43:56.798657 18824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:56.799722 18892 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:56.802047 18892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: Bootstrap starting.
I20260927 18:43:56.802665 18892 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:56.802961 18892 log.cc:824] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:56.803768 18892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: No bootstrap required, opened a new log
I20260927 18:43:56.805433 18892 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } }
I20260927 18:43:56.805583 18892 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:56.805606 18892 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0cce5ae2eca48b4a6e530c5ac668388, State: Initialized, Role: FOLLOWER
I20260927 18:43:56.805778 18892 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } }
I20260927 18:43:56.805845 18892 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:56.805866 18892 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:56.805894 18892 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:56.806744 18892 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } }
I20260927 18:43:56.806861 18892 leader_election.cc:302] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0cce5ae2eca48b4a6e530c5ac668388; no voters:
I20260927 18:43:56.807057 18892 leader_election.cc:288] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:56.807113 18895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:56.807332 18895 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [term 1 LEADER]: Becoming Leader. State: Replica: f0cce5ae2eca48b4a6e530c5ac668388, State: Running, Role: LEADER
I20260927 18:43:56.807366 18892 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:56.807423 18895 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } }
I20260927 18:43:56.807726 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18824
I20260927 18:43:56.807849 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:56.808033 15803 external_mini_cluster.cc:1428] Setting key 4e40b74d06e10fbf35ad400601f22581
I20260927 18:43:56.808636 18897 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0cce5ae2eca48b4a6e530c5ac668388. Latest consensus state: current_term: 1 leader_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } } }
I20260927 18:43:56.808749 18896 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0cce5ae2eca48b4a6e530c5ac668388" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 35259 } } }
I20260927 18:43:56.808876 18896 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:56.808754 18897 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388 [sys.catalog]: This master's current role is: LEADER
W20260927 18:43:56.809510 18910 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:43:56.809574 18910 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:43:56.809625 18913 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:56.809813 18913 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:56.811509 18913 catalog_manager.cc:1347] Generated new cluster ID: 1f00b0d43c224f4fb118772eccc1da7e
I20260927 18:43:56.811563 18913 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:43:56 dist-test-slave-vs05 krb5kdc[18811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534636, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:56.815744 18913 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:56.816376 18913 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:56.820147 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:56.808919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60610 (local address 127.15.110.254:35259)
0927 18:43:56.809156 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:56.809167 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:56.809368 (+ 201us) server_negotiation.cc:407] Connection header received
0927 18:43:56.809545 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:56.809553 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:56.809629 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:56.809809 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:56.810528 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:56.811869 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:56.812909 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:56.813318 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:56.813400 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:56.816174 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:56.816178 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:56.816179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:56.816192 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:56.818438 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:56.818860 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:56.818863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:56.818864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:56.818975 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:56.819225 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:56.819226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:56.819227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:56.819524 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:56.819691 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:56.819778 (+ 87us) server_negotiation.cc:299] Negotiation successful
0927 18:43:56.819933 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
I20260927 18:43:56.825932 18913 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0cce5ae2eca48b4a6e530c5ac668388: Generated new TSK 0
I20260927 18:43:56.826131 18913 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:56.856204 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:56.936894 18918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:56.937150 18918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:56.937189 18918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:56.938346 18918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:56.938416 18918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:56.938459 18918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:56.939996 18918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:56.940241 18918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:56.940418 18918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:56.945459 18924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:56.945472 18923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:56.945539 18926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:56.945760 18918 server_base.cc:1034] running on GCE node
I20260927 18:43:56.945979 18918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:56.946179 18918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:56.947314 18918 hybrid_clock.cc:648] HybridClock initialized: now 1790534636947286 us; error 40 us; skew 500 ppm
Sep 27 18:43:56 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:56.949963 18918 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:56.950492 18918 webserver.cc:460] Webserver started at http://127.15.110.193:42807/ using document root <none> and password file <none>
I20260927 18:43:56.950727 18918 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:56.950798 18918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:56.950953 18918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:56.952167 18918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "b415efa5e4ef4522878a23aeb03b0b21"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "fc20b5fd90a91af893bfb626b896e88e"
server_key_iv: "9179b49f17e8bfc3c456a3f59a956000"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.952553 18918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "b415efa5e4ef4522878a23aeb03b0b21"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "fc20b5fd90a91af893bfb626b896e88e"
server_key_iv: "9179b49f17e8bfc3c456a3f59a956000"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.954090 18918 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:56.954869 18933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:56.955034 18918 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:56.955133 18918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "b415efa5e4ef4522878a23aeb03b0b21"
format_stamp: "Formatted at 2026-09-27 18:43:56 on dist-test-slave-vs05"
server_key: "fc20b5fd90a91af893bfb626b896e88e"
server_key_iv: "9179b49f17e8bfc3c456a3f59a956000"
server_key_version: "encryptionkey@0"
I20260927 18:43:56.955250 18918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:56.986182 18918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:56.988750 18918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:56.988929 18918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:56.989195 18918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:56.989543 18918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:56.989598 18918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:56.989631 18918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:56.989660 18918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:56.995654 18918 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:38105
I20260927 18:43:56.995699 19046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:38105 every 8 connection(s)
I20260927 18:43:56.996038 18918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:57.001852 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 18918
I20260927 18:43:57.001956 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:57.002166 15803 external_mini_cluster.cc:1428] Setting key d60a9fd7ba8330d2b9959c0c92bcc2a4
Sep 27 18:43:56 dist-test-slave-vs05 krb5kdc[18811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534636, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:57.006394 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:56.996818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:44339 (local address 127.15.110.254:35259)
0927 18:43:56.996873 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:56.996875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:56.997101 (+ 226us) server_negotiation.cc:407] Connection header received
0927 18:43:56.997570 (+ 469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:56.997571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:56.997582 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:56.997639 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:56.998561 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:56.999051 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.000214 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.000409 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.000467 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.002946 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:57.002947 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:57.002948 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:57.002956 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:57.004505 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.005025 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.005025 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.005026 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.005086 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.005366 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.005367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.005367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.005470 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:57.005581 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:57.006022 (+ 441us) server_negotiation.cc:299] Negotiation successful
0927 18:43:57.006182 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:43:57.006457 19049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:56.996791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35259 (local address 127.15.110.193:44339)
0927 18:43:56.996961 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:56.997003 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:56.997486 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:56.997705 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:56.997708 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:56.998270 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:56.998480 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:56.998485 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:56.999165 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:56.999166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.000116 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:57.000118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.000487 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.000488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.000595 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.001056 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:57.001065 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:57.002840 (+ 1775us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:57.004594 (+ 1754us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.004603 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.004605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.004934 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.005153 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.005154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.005155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.005301 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.005632 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:57.005641 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:57.005823 (+ 182us) client_negotiation.cc:769] Sending connection context
0927 18:43:57.005945 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 18:43:57.006168 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260927 18:43:57.007220 19047 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35259
I20260927 18:43:57.007391 19047 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:57.007614 19047 heartbeater.cc:511] Master 127.15.110.254:35259 requested a full tablet report, sending...
I20260927 18:43:57.008281 18856 ts_manager.cc:194] Registered new tserver with Master: b415efa5e4ef4522878a23aeb03b0b21 (127.15.110.193:38105)
I20260927 18:43:57.009298 18856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:44339
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:57.030083 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:57.114223 19052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:57.114419 19052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:57.114442 19052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:57.115614 19052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:57.115657 19052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:57.115684 19052 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:57.117067 19052 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:57.117246 19052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:57.117415 19052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:57.122334 19058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:57.122409 19060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:57.122454 19052 server_base.cc:1034] running on GCE node
W20260927 18:43:57.122339 19057 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:57.122833 19052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:57.123031 19052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:57.124197 19052 hybrid_clock.cc:648] HybridClock initialized: now 1790534637124168 us; error 39 us; skew 500 ppm
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:57.126829 19052 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:57.127274 19052 webserver.cc:460] Webserver started at http://127.15.110.194:35971/ using document root <none> and password file <none>
I20260927 18:43:57.127462 19052 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:57.127506 19052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:57.127600 19052 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:57.128592 19052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "d5688b04d4c644e490269f488a1e7785"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "6d8f1dc697c6338db93695d9fddac28f"
server_key_iv: "97cb934f23302fa76e8e76e6c8e2780d"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.128895 19052 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "d5688b04d4c644e490269f488a1e7785"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "6d8f1dc697c6338db93695d9fddac28f"
server_key_iv: "97cb934f23302fa76e8e76e6c8e2780d"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.130199 19052 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:57.130934 19067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.131124 19052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:43:57.131255 19052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "d5688b04d4c644e490269f488a1e7785"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "6d8f1dc697c6338db93695d9fddac28f"
server_key_iv: "97cb934f23302fa76e8e76e6c8e2780d"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.131321 19052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:57.161619 19052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:57.164139 19052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:57.164250 19052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:57.164431 19052 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:57.164755 19052 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:57.164789 19052 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.164850 19052 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:57.164876 19052 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.170729 19052 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:39563
I20260927 18:43:57.170792 19180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:39563 every 8 connection(s)
I20260927 18:43:57.171130 19052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:57.175122 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19052
I20260927 18:43:57.175284 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:57.175477 15803 external_mini_cluster.cc:1428] Setting key 47a537ecbdec19a7931cbff3d7f0e8a5
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:57.181067 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:57.171931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54691 (local address 127.15.110.254:35259)
0927 18:43:57.171974 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:57.171976 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:57.172207 (+ 231us) server_negotiation.cc:407] Connection header received
0927 18:43:57.172744 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:57.172745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:57.172757 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:57.172814 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:57.173779 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.174279 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.175340 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.175598 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.175688 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.178073 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:57.178074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:57.178074 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:57.178091 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:57.179586 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.180080 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.180080 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.180081 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.180131 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.180414 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.180414 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.180415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.180528 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:57.180583 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:57.180935 (+ 352us) server_negotiation.cc:299] Negotiation successful
0927 18:43:57.181004 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:43:57.181391 19183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:57.171897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35259 (local address 127.15.110.194:54691)
0927 18:43:57.172082 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:57.172119 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:57.172637 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:57.172886 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:57.172890 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:57.173461 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:57.173668 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:57.173672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.174374 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.174375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.175254 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:57.175257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.175696 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.175697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.175846 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.176376 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:57.176384 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:57.177964 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:57.179664 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.179671 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.179673 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.180000 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.180198 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.180199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.180199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.180342 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.180601 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:57.180606 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:57.180789 (+ 183us) client_negotiation.cc:769] Sending connection context
0927 18:43:57.180918 (+ 129us) client_negotiation.cc:240] Negotiation successful
0927 18:43:57.181103 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":98,"threads_started":1}
I20260927 18:43:57.181936 19181 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35259
I20260927 18:43:57.182027 19181 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:57.182216 19181 heartbeater.cc:511] Master 127.15.110.254:35259 requested a full tablet report, sending...
I20260927 18:43:57.182590 18856 ts_manager.cc:194] Registered new tserver with Master: d5688b04d4c644e490269f488a1e7785 (127.15.110.194:39563)
I20260927 18:43:57.182914 18856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:54691
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:57.201498 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:57.283094 19186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:57.283368 19186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:57.283394 19186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:57.284487 19186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:57.284521 19186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:57.284547 19186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:57.285979 19186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:35259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:57.286159 19186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:57.286325 19186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:57.291235 19194 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:57.291298 19192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:57.291326 19191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:57.291435 19186 server_base.cc:1034] running on GCE node
I20260927 18:43:57.291718 19186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:57.291962 19186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:57.293128 19186 hybrid_clock.cc:648] HybridClock initialized: now 1790534637293130 us; error 43 us; skew 500 ppm
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:57.295859 19186 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:57.296290 19186 webserver.cc:460] Webserver started at http://127.15.110.195:42387/ using document root <none> and password file <none>
I20260927 18:43:57.296525 19186 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:57.296597 19186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:57.296731 19186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:57.297763 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "dc47091788ce4e5aa36d66578325c293"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "de7b86841f88529b33e3719ca72f0a4a"
server_key_iv: "c1348f20b771e1cca93b28ba8da540de"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.298131 19186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "dc47091788ce4e5aa36d66578325c293"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "de7b86841f88529b33e3719ca72f0a4a"
server_key_iv: "c1348f20b771e1cca93b28ba8da540de"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.299511 19186 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:57.300273 19201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.300463 19186 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:43:57.300561 19186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "dc47091788ce4e5aa36d66578325c293"
format_stamp: "Formatted at 2026-09-27 18:43:57 on dist-test-slave-vs05"
server_key: "de7b86841f88529b33e3719ca72f0a4a"
server_key_iv: "c1348f20b771e1cca93b28ba8da540de"
server_key_version: "encryptionkey@0"
I20260927 18:43:57.300645 19186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:57.307134 19186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:57.309406 19186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:57.309571 19186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:57.309798 19186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:57.310096 19186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:57.310151 19186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.310194 19186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:57.310220 19186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:57.316207 19186 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40565
I20260927 18:43:57.316255 19314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40565 every 8 connection(s)
I20260927 18:43:57.316622 19186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:57.325888 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:57.317367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:54225 (local address 127.15.110.254:35259)
0927 18:43:57.317434 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:57.317437 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:57.317657 (+ 220us) server_negotiation.cc:407] Connection header received
0927 18:43:57.318137 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:57.318138 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:57.318149 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:57.318203 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:57.319225 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.319756 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.320746 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.320937 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.320994 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.323171 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:57.323173 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:57.323212 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:57.323220 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:57.324423 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.324890 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.324890 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.324891 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.324942 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.325182 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.325183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.325183 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.325284 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:57.325333 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:57.325658 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 18:43:57.325751 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:43:57.325925 19317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:57.317357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:35259 (local address 127.15.110.195:54225)
0927 18:43:57.317524 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:57.317561 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:57.318043 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:57.318257 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:57.318260 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:57.318865 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:57.319065 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:57.319071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.319842 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.319843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.320669 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:57.320671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.321015 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.321016 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:57.321117 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.321672 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:57.321681 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:57.323087 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:57.324560 (+ 1473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.324566 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.324569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.324816 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.325016 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:57.325016 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:57.325017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:57.325124 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:57.325342 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:57.325346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:57.325470 (+ 124us) client_negotiation.cc:769] Sending connection context
0927 18:43:57.325618 (+ 148us) client_negotiation.cc:240] Negotiation successful
0927 18:43:57.325746 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":103,"threads_started":1}
I20260927 18:43:57.326303 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19186
I20260927 18:43:57.326386 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:57.326553 15803 external_mini_cluster.cc:1428] Setting key f451acae35a278b119c95bb68d052060
I20260927 18:43:57.326587 19315 heartbeater.cc:347] Connected to a master server at 127.15.110.254:35259
I20260927 18:43:57.326680 19315 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:57.326876 19315 heartbeater.cc:511] Master 127.15.110.254:35259 requested a full tablet report, sending...
I20260927 18:43:57.327310 18856 ts_manager.cc:194] Registered new tserver with Master: dc47091788ce4e5aa36d66578325c293 (127.15.110.195:40565)
I20260927 18:43:57.327564 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:43:57.327881 18856 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:54225
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[18811](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534637, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:57.350383 18912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:57.343014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60612 (local address 127.15.110.254:35259)
0927 18:43:57.343108 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:57.343110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:57.343222 (+ 112us) server_negotiation.cc:407] Connection header received
0927 18:43:57.343312 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:57.343313 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:57.343329 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:57.343392 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:57.344005 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.344515 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.345303 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:57.345497 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:57.345580 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:57.347863 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:57.347864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:57.347865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:57.347878 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:57.349137 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.349480 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.349482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.349482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.349575 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:57.349791 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:57.349792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:57.349792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:57.349951 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:57.350078 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:57.350183 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:43:57.350268 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260927 18:43:57.350682 18856 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:60612
I20260927 18:43:57.351156 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18918
I20260927 18:43:57.355705 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19052
I20260927 18:43:57.360375 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19186
I20260927 18:43:57.364614 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 18824
2026-09-27T18:43:57Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1861 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[19327](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[19327](info): set up 2 sockets
Sep 27 18:43:57 dist-test-slave-vs05 krb5kdc[19327](info): commencing operation
krb5kdc: starting...
W20260927 18:43:58.407073 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:43:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:43:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:43:58.526296 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41247
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:41247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:58.607734 19340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:58.607996 19340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:58.608035 19340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:58.609184 19340 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:43:58.609261 19340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:58.609284 19340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:58.609306 19340 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:43:58.609334 19340 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:43:58.610838 19340 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:41247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:58.611123 19340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:58.611373 19340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:58.616338 19345 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:58.616394 19348 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:58.616353 19346 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:58.616639 19340 server_base.cc:1034] running on GCE node
I20260927 18:43:58.617005 19340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:58.617298 19340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:58.618443 19340 hybrid_clock.cc:648] HybridClock initialized: now 1790534638618421 us; error 44 us; skew 500 ppm
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:58.621004 19340 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:43:58.621465 19340 webserver.cc:460] Webserver started at http://127.15.110.254:35303/ using document root <none> and password file <none>
I20260927 18:43:58.621701 19340 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:58.621771 19340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:58.621927 19340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:58.623072 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "c37748dee3ea407080d98c2a3c78b9e6"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "371d2806f5073bc4899d06f6609b1c57"
server_key_iv: "1ba5d36d328f73ef1a23ac01c9e3aa56"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.623500 19340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "c37748dee3ea407080d98c2a3c78b9e6"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "371d2806f5073bc4899d06f6609b1c57"
server_key_iv: "1ba5d36d328f73ef1a23ac01c9e3aa56"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.624856 19340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:58.625602 19355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:58.625777 19340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:43:58.625885 19340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "c37748dee3ea407080d98c2a3c78b9e6"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "371d2806f5073bc4899d06f6609b1c57"
server_key_iv: "1ba5d36d328f73ef1a23ac01c9e3aa56"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.625960 19340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:58.648944 19340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:58.651461 19340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:58.651674 19340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:58.656178 19340 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:41247
I20260927 18:43:58.656230 19407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:41247 every 8 connection(s)
I20260927 18:43:58.656756 19340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:43:58.657454 19408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:43:58.660763 19408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6: Bootstrap starting.
I20260927 18:43:58.661273 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19340
I20260927 18:43:58.661391 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:43:58.661603 15803 external_mini_cluster.cc:1428] Setting key 1d37022cdf2d11eea3b72cdc4ab1367d
I20260927 18:43:58.661614 19408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6: Neither blocks nor log segments found. Creating new log.
I20260927 18:43:58.661914 19408 log.cc:824] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:43:58.662964 19408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6: No bootstrap required, opened a new log
I20260927 18:43:58.665030 19408 raft_consensus.cc:348] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } }
I20260927 18:43:58.665205 19408 raft_consensus.cc:374] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:43:58.665253 19408 raft_consensus.cc:729] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37748dee3ea407080d98c2a3c78b9e6, State: Initialized, Role: FOLLOWER
I20260927 18:43:58.665437 19408 consensus_queue.cc:260] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } }
I20260927 18:43:58.665537 19408 raft_consensus.cc:388] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:43:58.665587 19408 raft_consensus.cc:482] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:43:58.665637 19408 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:43:58.666209 19408 raft_consensus.cc:504] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } }
I20260927 18:43:58.666330 19408 leader_election.cc:302] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c37748dee3ea407080d98c2a3c78b9e6; no voters:
I20260927 18:43:58.666548 19408 leader_election.cc:288] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:43:58.666570 19413 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:43:58.666745 19413 raft_consensus.cc:686] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [term 1 LEADER]: Becoming Leader. State: Replica: c37748dee3ea407080d98c2a3c78b9e6, State: Running, Role: LEADER
I20260927 18:43:58.666877 19408 sys_catalog.cc:564] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:43:58.666916 19413 consensus_queue.cc:237] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } }
I20260927 18:43:58.667349 19414 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c37748dee3ea407080d98c2a3c78b9e6. Latest consensus state: current_term: 1 leader_uuid: "c37748dee3ea407080d98c2a3c78b9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } } }
I20260927 18:43:58.667531 19414 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:58.667855 19422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:43:58.668434 19422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:43:58.668390 19415 sys_catalog.cc:455] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c37748dee3ea407080d98c2a3c78b9e6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c37748dee3ea407080d98c2a3c78b9e6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41247 } } }
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:58.668653 19415 sys_catalog.cc:458] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:43:58.670341 19422 catalog_manager.cc:1347] Generated new cluster ID: c3f8260b194948f8afe58833526d3b80
I20260927 18:43:58.670395 19422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:43:58.673144 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:58.662143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48060 (local address 127.15.110.254:41247)
0927 18:43:58.662364 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:58.662375 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:58.662426 (+ 51us) server_negotiation.cc:407] Connection header received
0927 18:43:58.662967 (+ 541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:58.662975 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:58.663030 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:58.663229 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:58.663877 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.664813 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.665772 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.666214 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.666342 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:58.669399 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:58.669403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:58.669403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:58.669416 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:58.671527 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:58.671907 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:58.671910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:58.671911 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:58.672014 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:58.672242 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:58.672243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:58.672243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:58.672499 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:58.672705 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:58.672782 (+ 77us) server_negotiation.cc:299] Negotiation successful
0927 18:43:58.672920 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":90,"threads_started":1}
I20260927 18:43:58.688434 19422 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:43:58.689206 19422 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:43:58.696813 19422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c37748dee3ea407080d98c2a3c78b9e6: Generated new TSK 0
I20260927 18:43:58.697055 19422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:43:58.727105 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:58.809583 19434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:58.809794 19434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:58.809818 19434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:58.810909 19434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:58.810956 19434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:58.810983 19434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:43:58.812439 19434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:58.812721 19434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:58.812929 19434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:58.818203 19439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:58.818287 19434 server_base.cc:1034] running on GCE node
W20260927 18:43:58.818202 19440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:58.818267 19442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:58.818711 19434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:58.818969 19434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:58.820151 19434 hybrid_clock.cc:648] HybridClock initialized: now 1790534638820128 us; error 40 us; skew 500 ppm
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:58.823011 19434 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:43:58.823570 19434 webserver.cc:460] Webserver started at http://127.15.110.193:36687/ using document root <none> and password file <none>
I20260927 18:43:58.823798 19434 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:58.823868 19434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:58.824034 19434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:58.825152 19434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "0c215c861fe449f696df9760d45c0b48"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "eaccdc08cbd3a041424e8734ccabbbe1"
server_key_iv: "f8c7635f59f9487acf673529856e34fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.825532 19434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "0c215c861fe449f696df9760d45c0b48"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "eaccdc08cbd3a041424e8734ccabbbe1"
server_key_iv: "f8c7635f59f9487acf673529856e34fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.826979 19434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:43:58.827924 19449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:58.828136 19434 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:58.828244 19434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "0c215c861fe449f696df9760d45c0b48"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "eaccdc08cbd3a041424e8734ccabbbe1"
server_key_iv: "f8c7635f59f9487acf673529856e34fa"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.828325 19434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:58.837980 19434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:58.840425 19434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:58.840543 19434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:58.840731 19434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:58.840998 19434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:58.841028 19434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:58.841049 19434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:58.841063 19434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:58.846802 19434 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:43415
I20260927 18:43:58.846877 19562 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:43415 every 8 connection(s)
I20260927 18:43:58.847259 19434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:43:58.852361 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19434
I20260927 18:43:58.852535 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:43:58.852730 15803 external_mini_cluster.cc:1428] Setting key c0e6f622e1f98a6b6864ad1ee68191cb
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:58.858323 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:58.848113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39447 (local address 127.15.110.254:41247)
0927 18:43:58.848186 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:58.848204 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:58.848403 (+ 199us) server_negotiation.cc:407] Connection header received
0927 18:43:58.849401 (+ 998us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:58.849402 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:58.849415 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:58.849463 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:58.850103 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.850624 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.851755 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.851981 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.852071 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:58.855037 (+ 2966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:58.855039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:58.855039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:58.855049 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:58.856633 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:58.857169 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:58.857170 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:58.857170 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:58.857251 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:58.857623 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:58.857624 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:58.857625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:58.857750 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:58.857806 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:58.858184 (+ 378us) server_negotiation.cc:299] Negotiation successful
0927 18:43:58.858255 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:43:58.858654 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:58.848042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41247 (local address 127.15.110.193:39447)
0927 18:43:58.848277 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:58.848313 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:58.849278 (+ 965us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:58.849537 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:58.849541 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:58.849805 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:58.849996 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:58.850000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.850722 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.850723 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:58.851646 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:58.851648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:58.852082 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:58.852083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:58.852210 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:58.852776 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:58.852785 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:58.854891 (+ 2106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:58.856730 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:58.856737 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:58.856739 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:58.857089 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:58.857332 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:58.857333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:58.857334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:58.857505 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:58.857824 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:58.857830 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:58.858012 (+ 182us) client_negotiation.cc:769] Sending connection context
0927 18:43:58.858169 (+ 157us) client_negotiation.cc:240] Negotiation successful
0927 18:43:58.858362 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":116,"threads_started":1}
I20260927 18:43:58.859304 19563 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41247
I20260927 18:43:58.859401 19563 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:58.859613 19563 heartbeater.cc:511] Master 127.15.110.254:41247 requested a full tablet report, sending...
I20260927 18:43:58.860183 19372 ts_manager.cc:194] Registered new tserver with Master: 0c215c861fe449f696df9760d45c0b48 (127.15.110.193:43415)
I20260927 18:43:58.860790 19372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:39447
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:43:58.878696 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:58.964205 19568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:58.964458 19568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:58.964497 19568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:58.965682 19568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:58.965757 19568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:58.965802 19568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:43:58.967623 19568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:58.967902 19568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:58.968226 19568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:58.973423 19574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:58.973423 19576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:58.973460 19573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:58.973946 19568 server_base.cc:1034] running on GCE node
I20260927 18:43:58.974203 19568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:58.974428 19568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:58.975569 19568 hybrid_clock.cc:648] HybridClock initialized: now 1790534638975556 us; error 32 us; skew 500 ppm
Sep 27 18:43:58 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:58.978228 19568 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:43:58.978740 19568 webserver.cc:460] Webserver started at http://127.15.110.194:33829/ using document root <none> and password file <none>
I20260927 18:43:58.978991 19568 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:58.979066 19568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:58.979259 19568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:58.980358 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "0e092131eb494d54a1331ceb31103f36"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "2e4d07487fa5bf5d09dfd081752538f0"
server_key_iv: "5f0684fb0b81c5e5ab4b9316f0c197fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.980726 19568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "0e092131eb494d54a1331ceb31103f36"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "2e4d07487fa5bf5d09dfd081752538f0"
server_key_iv: "5f0684fb0b81c5e5ab4b9316f0c197fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.982052 19568 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:58.982880 19583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:58.983060 19568 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:58.983158 19568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "0e092131eb494d54a1331ceb31103f36"
format_stamp: "Formatted at 2026-09-27 18:43:58 on dist-test-slave-vs05"
server_key: "2e4d07487fa5bf5d09dfd081752538f0"
server_key_iv: "5f0684fb0b81c5e5ab4b9316f0c197fd"
server_key_version: "encryptionkey@0"
I20260927 18:43:58.983260 19568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:59.001332 19568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:59.003765 19568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:59.003934 19568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:59.004176 19568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:59.004498 19568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:59.004550 19568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:59.004601 19568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:59.004628 19568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:59.012776 19568 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:45905
I20260927 18:43:59.012845 19696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:45905 every 8 connection(s)
I20260927 18:43:59.013336 19568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:43:59.013607 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19568
I20260927 18:43:59.013715 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:43:59.014573 15803 external_mini_cluster.cc:1428] Setting key 04672d62558f957723f5faab5f0f12da
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534638, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
I20260927 18:43:59.026345 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.015275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54255 (local address 127.15.110.254:41247)
0927 18:43:59.015359 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:59.015362 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:59.015436 (+ 74us) server_negotiation.cc:407] Connection header received
0927 18:43:59.016666 (+ 1230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:59.016668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:59.016681 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:59.016730 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:59.017432 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.018156 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.019901 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.020098 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.020172 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.023046 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:59.023047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:59.023048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:59.023064 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:59.024800 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.025271 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.025272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.025273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.025333 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.025636 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.025637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.025638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.025769 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:59.025849 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:59.026159 (+ 310us) server_negotiation.cc:299] Negotiation successful
0927 18:43:59.026242 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260927 18:43:59.026563 19699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.015001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41247 (local address 127.15.110.194:54255)
0927 18:43:59.015322 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:59.015371 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:59.016537 (+ 1166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:59.016851 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:59.016855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:59.017133 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:59.017330 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:59.017335 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.018260 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.018261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:59.019790 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:59.019793 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.020197 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.020198 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:59.020373 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.021026 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:59.021039 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:59.022923 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:59.024909 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:59.024917 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:59.024920 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:59.025175 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:59.025414 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:59.025415 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:59.025415 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:59.025563 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:59.025840 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:59.025847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:59.026002 (+ 155us) client_negotiation.cc:769] Sending connection context
0927 18:43:59.026136 (+ 134us) client_negotiation.cc:240] Negotiation successful
0927 18:43:59.026308 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":82,"threads_started":1}
I20260927 18:43:59.027380 19697 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41247
I20260927 18:43:59.027493 19697 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:59.027691 19697 heartbeater.cc:511] Master 127.15.110.254:41247 requested a full tablet report, sending...
I20260927 18:43:59.028119 19372 ts_manager.cc:194] Registered new tserver with Master: 0e092131eb494d54a1331ceb31103f36 (127.15.110.194:45905)
I20260927 18:43:59.028514 19372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:54255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:43:59.042980 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:34721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:43:59.124483 19702 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:43:59.124691 19702 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:43:59.124714 19702 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:43:59.126042 19702 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:43:59.126093 19702 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:43:59.126120 19702 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:43:59.127764 19702 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:34721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19702
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:43:59.128007 19702 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:43:59.128206 19702 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:43:59.133656 19710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:59.133656 19707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:43:59.133682 19708 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:43:59.133814 19702 server_base.cc:1034] running on GCE node
I20260927 18:43:59.134178 19702 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:43:59.134402 19702 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:43:59.135576 19702 hybrid_clock.cc:648] HybridClock initialized: now 1790534639135545 us; error 45 us; skew 500 ppm
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534639, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:43:59.138273 19702 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:43:59.138926 19702 webserver.cc:460] Webserver started at http://127.15.110.195:33525/ using document root <none> and password file <none>
I20260927 18:43:59.139266 19702 fs_manager.cc:359] Metadata directory not provided
I20260927 18:43:59.139354 19702 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:43:59.139490 19702 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:43:59.140592 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "73e0f4f397f042dcaf018034b24509c9"
format_stamp: "Formatted at 2026-09-27 18:43:59 on dist-test-slave-vs05"
server_key: "5c745854c599eb4630f941de88f9f6a4"
server_key_iv: "e89e4cfa10387d2541edaa4bd9fc111b"
server_key_version: "encryptionkey@0"
I20260927 18:43:59.140952 19702 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "73e0f4f397f042dcaf018034b24509c9"
format_stamp: "Formatted at 2026-09-27 18:43:59 on dist-test-slave-vs05"
server_key: "5c745854c599eb4630f941de88f9f6a4"
server_key_iv: "e89e4cfa10387d2541edaa4bd9fc111b"
server_key_version: "encryptionkey@0"
I20260927 18:43:59.142311 19702 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:43:59.143280 19717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:59.143512 19702 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:43:59.143617 19702 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "73e0f4f397f042dcaf018034b24509c9"
format_stamp: "Formatted at 2026-09-27 18:43:59 on dist-test-slave-vs05"
server_key: "5c745854c599eb4630f941de88f9f6a4"
server_key_iv: "e89e4cfa10387d2541edaa4bd9fc111b"
server_key_version: "encryptionkey@0"
I20260927 18:43:59.143707 19702 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:43:59.154174 19702 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:43:59.156510 19702 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:43:59.156742 19702 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:43:59.156988 19702 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:43:59.157315 19702 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:43:59.157371 19702 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:59.157414 19702 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:43:59.157447 19702 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:43:59.163697 19702 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:45301
I20260927 18:43:59.163738 19830 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:45301 every 8 connection(s)
I20260927 18:43:59.164072 19702 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:43:59.168349 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19702
I20260927 18:43:59.168488 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:43:59.168751 15803 external_mini_cluster.cc:1428] Setting key 765e727eefb3c16c1ad36bf4a2d3dc8e
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534639, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:59.175066 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.164832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:40657 (local address 127.15.110.254:41247)
0927 18:43:59.164875 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:59.164877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:59.165106 (+ 229us) server_negotiation.cc:407] Connection header received
0927 18:43:59.166134 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:59.166135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:59.166146 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:59.166203 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:59.166910 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.167679 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.169261 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.169694 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.169782 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.172191 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:59.172192 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:59.172193 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:59.172200 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:59.173715 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.174114 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.174115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.174116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.174190 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.174431 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.174432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.174432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.174540 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:59.174640 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:59.174916 (+ 276us) server_negotiation.cc:299] Negotiation successful
0927 18:43:59.174995 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:43:59.175248 19833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.164806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41247 (local address 127.15.110.195:40657)
0927 18:43:59.164963 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 18:43:59.165000 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:43:59.166009 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:43:59.166263 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:43:59.166267 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:43:59.166528 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:43:59.166808 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:59.166813 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.167794 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.167795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:59.169151 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:43:59.169154 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.169816 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.169818 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:43:59.169970 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.170496 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:43:59.170504 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:43:59.172079 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:43:59.173794 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:59.173801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:59.173803 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:59.174041 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:59.174252 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:43:59.174252 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:43:59.174253 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:43:59.174374 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:43:59.174644 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:43:59.174648 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:43:59.174773 (+ 125us) client_negotiation.cc:769] Sending connection context
0927 18:43:59.174859 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:43:59.174990 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":95,"threads_started":1}
I20260927 18:43:59.175851 19831 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41247
I20260927 18:43:59.175961 19831 heartbeater.cc:464] Registering TS with master...
I20260927 18:43:59.176118 19831 heartbeater.cc:511] Master 127.15.110.254:41247 requested a full tablet report, sending...
I20260927 18:43:59.176589 19372 ts_manager.cc:194] Registered new tserver with Master: 73e0f4f397f042dcaf018034b24509c9 (127.15.110.195:45301)
I20260927 18:43:59.176945 19372 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:40657
I20260927 18:43:59.180936 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:43:59.205310 19838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.196583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44278 (local address 127.15.110.193:43415)
0927 18:43:59.196816 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:59.196823 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:59.196837 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:43:59.196897 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:59.196899 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:59.196915 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:59.196983 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:59.197695 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.198450 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.199362 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.199785 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.199863 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.202301 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:59.202311 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:59.202311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:59.202319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:59.204170 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.204548 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.204551 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.204551 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.204623 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.204804 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.204804 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.204804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.204937 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:59.205049 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:59.205122 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 18:43:59.205183 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":108,"threads_started":1}
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19327](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534639, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:43:59.214432 19412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:43:59.206803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48064 (local address 127.15.110.254:41247)
0927 18:43:59.206853 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:43:59.206855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:43:59.206893 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:43:59.206948 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:43:59.206949 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:43:59.206957 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:43:59.207002 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:43:59.207703 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.208225 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.209004 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:43:59.209221 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:43:59.209286 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:43:59.211751 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:43:59.211752 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:43:59.211753 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:43:59.211761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:43:59.213167 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.213541 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.213542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.213542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.213603 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:43:59.213805 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:43:59.213806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:43:59.213806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:43:59.213961 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:43:59.214088 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0927 18:43:59.214249 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 18:43:59.214325 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:43:59.214712 19372 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48064
I20260927 18:43:59.215099 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19434
I20260927 18:43:59.219624 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19568
I20260927 18:43:59.223827 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19702
I20260927 18:43:59.228343 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19340
2026-09-27T18:43:59Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1864 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19843](info): set up 2 sockets
Sep 27 18:43:59 dist-test-slave-vs05 krb5kdc[19843](info): commencing operation
krb5kdc: starting...
W20260927 18:44:00.267321 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:44:00.386435 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:34639
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:34639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:00.466194 19856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:00.466431 19856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:00.466470 19856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:00.467651 19856 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:00.467721 19856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:00.467746 19856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:00.467774 19856 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:00.467805 19856 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:00.469300 19856 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:34639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:34639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:00.469578 19856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:00.469805 19856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:00.474797 19864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:00.474758 19861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.474826 19856 server_base.cc:1034] running on GCE node
W20260927 18:44:00.474773 19862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.475308 19856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:00.475594 19856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:00.476761 19856 hybrid_clock.cc:648] HybridClock initialized: now 1790534640476745 us; error 28 us; skew 500 ppm
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:00.484634 19856 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:44:00.485008 19856 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:00.485059 19856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:00.485152 19856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:00.486234 19856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "0fa47574a8de4e5e94d9288eb4be7c44"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "5e0538294193663862cabe886a9a7c38"
server_key_iv: "187d3d8b1d6ced2c6625a00393c913ec"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.486541 19856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "0fa47574a8de4e5e94d9288eb4be7c44"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "5e0538294193663862cabe886a9a7c38"
server_key_iv: "187d3d8b1d6ced2c6625a00393c913ec"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.487870 19856 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:44:00.488592 19870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.488780 19856 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:00.488867 19856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "0fa47574a8de4e5e94d9288eb4be7c44"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "5e0538294193663862cabe886a9a7c38"
server_key_iv: "187d3d8b1d6ced2c6625a00393c913ec"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.488924 19856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:00.527817 19856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:00.530712 19856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:00.530880 19856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:00.536509 19856 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:34639
I20260927 18:44:00.536566 19922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:34639 every 8 connection(s)
I20260927 18:44:00.536868 19856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:00.538092 19923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:00.541222 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19856
I20260927 18:44:00.541333 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:00.541509 15803 external_mini_cluster.cc:1428] Setting key 742f12036bb94c1248e094a240b05612
I20260927 18:44:00.542187 19923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: Bootstrap starting.
I20260927 18:44:00.543025 19923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:00.543478 19923 log.cc:824] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:00.544350 19923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: No bootstrap required, opened a new log
I20260927 18:44:00.547358 19923 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } }
I20260927 18:44:00.547669 19923 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:00.547736 19923 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa47574a8de4e5e94d9288eb4be7c44, State: Initialized, Role: FOLLOWER
I20260927 18:44:00.547911 19923 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } }
I20260927 18:44:00.548013 19923 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:00.548054 19923 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:00.548132 19923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:00.548893 19923 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } }
I20260927 18:44:00.549059 19923 leader_election.cc:302] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fa47574a8de4e5e94d9288eb4be7c44; no voters:
I20260927 18:44:00.549252 19923 leader_election.cc:288] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:00.549299 19928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:00.549485 19928 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa47574a8de4e5e94d9288eb4be7c44, State: Running, Role: LEADER
I20260927 18:44:00.549510 19923 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:00.549640 19928 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } }
I20260927 18:44:00.551290 19929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } } }
I20260927 18:44:00.551443 19929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [sys.catalog]: This master's current role is: LEADER
W20260927 18:44:00.551525 19942 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:44:00.551631 19942 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:44:00.551767 19944 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:00.551950 19944 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:00.553848 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:00.542042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33882 (local address 127.15.110.254:34639)
0927 18:44:00.542244 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:00.542254 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:00.542297 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:44:00.542718 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:00.542724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:00.542770 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:00.542924 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:00.543686 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.544810 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.545620 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.546083 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.546202 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:00.548836 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:00.548840 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:00.548841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:00.548853 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:00.551829 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.552281 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.552284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.552285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.552384 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.552599 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.552600 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.552600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.552949 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:00.553095 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:00.553506 (+ 411us) server_negotiation.cc:299] Negotiation successful
0927 18:44:00.553649 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":87,"threads_started":1}
I20260927 18:44:00.554492 19930 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fa47574a8de4e5e94d9288eb4be7c44. Latest consensus state: current_term: 1 leader_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa47574a8de4e5e94d9288eb4be7c44" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34639 } } }
I20260927 18:44:00.554584 19930 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:00.555367 19944 catalog_manager.cc:1347] Generated new cluster ID: 635ec2c59d3a412f9063bd625b4b1940
I20260927 18:44:00.555413 19944 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:00.565084 19944 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:00.565619 19944 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:00.568920 19944 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: Generated new TSK 0
I20260927 18:44:00.569113 19944 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:44:00.599218 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:00.684659 19949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:00.684859 19949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:00.684883 19949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:00.686049 19949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:00.686090 19949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:00.686117 19949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:00.687695 19949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:00.687952 19949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:00.688184 19949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:00.693255 19954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.693327 19949 server_base.cc:1034] running on GCE node
W20260927 18:44:00.693248 19957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:00.693251 19955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.693758 19949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:00.694008 19949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:00.695161 19949 hybrid_clock.cc:648] HybridClock initialized: now 1790534640695144 us; error 43 us; skew 500 ppm
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:00.697800 19949 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:44:00.698158 19949 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:00.698235 19949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:00.698390 19949 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:00.699564 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "dfb49fd3d2c847cb868cde745e48028a"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "2c1388581320378a55d2194a7cbccd04"
server_key_iv: "453e94af9279426bcd6daa696aee659a"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.699928 19949 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "dfb49fd3d2c847cb868cde745e48028a"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "2c1388581320378a55d2194a7cbccd04"
server_key_iv: "453e94af9279426bcd6daa696aee659a"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.701269 19949 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:00.701988 19963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.702155 19949 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:00.702260 19949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "dfb49fd3d2c847cb868cde745e48028a"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "2c1388581320378a55d2194a7cbccd04"
server_key_iv: "453e94af9279426bcd6daa696aee659a"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.702332 19949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:00.710528 19949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:00.712337 19949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:00.712498 19949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:00.712713 19949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:00.713018 19949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:00.713078 19949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.713119 19949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:00.713147 19949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.718858 19949 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46163
I20260927 18:44:00.718892 20076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46163 every 8 connection(s)
I20260927 18:44:00.719156 19949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:00.723950 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 19949
I20260927 18:44:00.724067 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:00.724236 15803 external_mini_cluster.cc:1428] Setting key 0639a272390a1da07ff833605696e72e
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:00.728618 20079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:00.719857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:34639 (local address 127.15.110.193:39267)
0927 18:44:00.720012 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:00.720052 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:00.720524 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:00.720751 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:00.720754 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:00.721000 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:00.721554 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:00.721559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.722224 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.722225 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:00.723095 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:00.723098 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.723514 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.723515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:00.723619 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:00.724168 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:00.724176 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:00.725624 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:00.727303 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:00.727309 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:00.727310 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:00.727573 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:00.727787 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:00.727787 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:00.727787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:00.727882 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:00.728152 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:00.728157 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:00.728267 (+ 110us) client_negotiation.cc:769] Sending connection context
0927 18:44:00.728347 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:44:00.728461 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":97,"threads_started":1}
I20260927 18:44:00.728629 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:00.719911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39267 (local address 127.15.110.254:34639)
0927 18:44:00.719956 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:00.719958 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:00.720138 (+ 180us) server_negotiation.cc:407] Connection header received
0927 18:44:00.720611 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:00.720612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:00.720621 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:00.720678 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:00.721637 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.722143 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.723219 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.723430 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.723483 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:00.725716 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:00.725718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:00.725718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:00.725727 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:00.727231 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.727652 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.727653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.727653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.727718 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.727946 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.727946 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.727947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.728088 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:00.728159 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:00.728395 (+ 236us) server_negotiation.cc:299] Negotiation successful
0927 18:44:00.728497 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:00.729166 20077 heartbeater.cc:347] Connected to a master server at 127.15.110.254:34639
I20260927 18:44:00.729276 20077 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:00.729435 20077 heartbeater.cc:511] Master 127.15.110.254:34639 requested a full tablet report, sending...
I20260927 18:44:00.729935 19887 ts_manager.cc:194] Registered new tserver with Master: dfb49fd3d2c847cb868cde745e48028a (127.15.110.193:46163)
I20260927 18:44:00.730566 19887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:39267
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:44:00.750043 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:00.831504 20082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:00.831732 20082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:00.831758 20082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:00.832993 20082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:00.833040 20082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:00.833069 20082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:00.834698 20082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:00.834967 20082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:00.835227 20082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:00.840446 20088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:00.840448 20087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.840466 20082 server_base.cc:1034] running on GCE node
W20260927 18:44:00.840548 20090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:00.840896 20082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:00.841176 20082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:00.842356 20082 hybrid_clock.cc:648] HybridClock initialized: now 1790534640842352 us; error 39 us; skew 500 ppm
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:00.844980 20082 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:44:00.845386 20082 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:00.845458 20082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:00.845610 20082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:00.846719 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "91ac88decf66405382aa3f05ba757772"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "c4b33914b7f70870649d766e78b259e5"
server_key_iv: "56537b4eeeb8ca2739ecd35240d6de86"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.847107 20082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "91ac88decf66405382aa3f05ba757772"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "c4b33914b7f70870649d766e78b259e5"
server_key_iv: "56537b4eeeb8ca2739ecd35240d6de86"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.848528 20082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:00.849331 20096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.849568 20082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:00.849686 20082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "91ac88decf66405382aa3f05ba757772"
format_stamp: "Formatted at 2026-09-27 18:44:00 on dist-test-slave-vs05"
server_key: "c4b33914b7f70870649d766e78b259e5"
server_key_iv: "56537b4eeeb8ca2739ecd35240d6de86"
server_key_version: "encryptionkey@0"
I20260927 18:44:00.849781 20082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:00.876832 20082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:00.879252 20082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:00.879482 20082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:00.879760 20082 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:00.880137 20082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:00.880196 20082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.880234 20082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:00.880278 20082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:00.886535 20082 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:34515
I20260927 18:44:00.886590 20209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:34515 every 8 connection(s)
I20260927 18:44:00.886860 20082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:44:00 dist-test-slave-vs05 krb5kdc[19843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534640, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:00.895396 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20082
I20260927 18:44:00.895530 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:00.895712 15803 external_mini_cluster.cc:1428] Setting key ee99133e9ddd225a4eb75c44529873cf
I20260927 18:44:00.897104 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:00.887650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:45961 (local address 127.15.110.254:34639)
0927 18:44:00.887731 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:00.887733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:00.887948 (+ 215us) server_negotiation.cc:407] Connection header received
0927 18:44:00.888419 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:00.888420 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:00.888430 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:00.888530 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:00.889555 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.890064 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.891274 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.891516 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.891578 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:00.893998 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:00.894000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:00.894001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:00.894010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:00.895683 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.896112 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.896113 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.896113 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.896187 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:00.896416 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:00.896418 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:00.896418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:00.896545 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:00.896625 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:00.896918 (+ 293us) server_negotiation.cc:299] Negotiation successful
0927 18:44:00.897013 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:44:00.897294 20212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:00.887608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:34639 (local address 127.15.110.194:45961)
0927 18:44:00.887816 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:00.887856 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:00.888327 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:00.888602 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:00.888605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:00.888865 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:00.889378 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:00.889384 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.890167 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.890168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:00.891081 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:00.891084 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:00.891600 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:00.891601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:00.891708 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:00.892232 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:00.892240 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:00.893899 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:00.895771 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:00.895780 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:00.895782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:00.896055 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:00.896248 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:00.896249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:00.896249 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:00.896339 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:00.896623 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:00.896631 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:00.896768 (+ 137us) client_negotiation.cc:769] Sending connection context
0927 18:44:00.896885 (+ 117us) client_negotiation.cc:240] Negotiation successful
0927 18:44:00.897048 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":106,"threads_started":1}
I20260927 18:44:00.898008 20210 heartbeater.cc:347] Connected to a master server at 127.15.110.254:34639
I20260927 18:44:00.898134 20210 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:00.898299 20210 heartbeater.cc:511] Master 127.15.110.254:34639 requested a full tablet report, sending...
I20260927 18:44:00.898720 19887 ts_manager.cc:194] Registered new tserver with Master: 91ac88decf66405382aa3f05ba757772 (127.15.110.194:34515)
I20260927 18:44:00.899116 19887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:45961
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:44:00.922472 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:01.005389 20215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:01.005667 20215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:01.005705 20215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:01.007990 20215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:01.008054 20215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:01.008106 20215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:01.010627 20215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:34639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:01.010885 20215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:01.011163 20215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:01.016736 20220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.016821 20215 server_base.cc:1034] running on GCE node
W20260927 18:44:01.016752 20221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.016793 20223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.017203 20215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:01.017409 20215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:01.018750 20215 hybrid_clock.cc:648] HybridClock initialized: now 1790534641018735 us; error 25 us; skew 500 ppm
Sep 27 18:44:01 dist-test-slave-vs05 krb5kdc[19843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:01.021303 20215 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:44:01.021677 20215 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:01.021725 20215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:01.021814 20215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:01.022821 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "a02294a7f0d24b6991e5724dc5dcc67b"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "54c2cb8709c127250b6e0fd62d4739c9"
server_key_iv: "058b8bf7ca005e8f6b187921043ec565"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.023114 20215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "a02294a7f0d24b6991e5724dc5dcc67b"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "54c2cb8709c127250b6e0fd62d4739c9"
server_key_iv: "058b8bf7ca005e8f6b187921043ec565"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.024427 20215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:01.025156 20229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.025318 20215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:01.025385 20215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "a02294a7f0d24b6991e5724dc5dcc67b"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "54c2cb8709c127250b6e0fd62d4739c9"
server_key_iv: "058b8bf7ca005e8f6b187921043ec565"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.025437 20215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:01.033879 20215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:01.036263 20215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:01.036372 20215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:01.036556 20215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:01.036880 20215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:01.036912 20215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.036934 20215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:01.036947 20215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.042824 20215 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:45291
I20260927 18:44:01.042887 20342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:45291 every 8 connection(s)
I20260927 18:44:01.043166 20215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:01.047739 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20215
I20260927 18:44:01.047865 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:01.048048 15803 external_mini_cluster.cc:1428] Setting key 7ee8e1ad23eb0d0f214425fc076d13e3
Sep 27 18:44:01 dist-test-slave-vs05 krb5kdc[19843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534641, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:01.052966 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.043957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:41677 (local address 127.15.110.254:34639)
0927 18:44:01.044030 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.044032 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.044234 (+ 202us) server_negotiation.cc:407] Connection header received
0927 18:44:01.044716 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.044717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.044727 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.044780 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.045695 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.046215 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.047228 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.047448 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.047517 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.050147 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.050149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.050149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:01.050156 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.051565 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:01.051997 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:01.051999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:01.051999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:01.052061 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:01.052317 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:01.052318 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:01.052319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:01.052442 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.052513 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.052792 (+ 279us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.052874 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:44:01.053231 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.043934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:34639 (local address 127.15.110.195:41677)
0927 18:44:01.044103 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.044143 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.044629 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.044841 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.044844 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.045100 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.045589 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.045594 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.046301 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.046302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.047094 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.047096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.047545 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.047547 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.047726 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.048329 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:01.048340 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.050038 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.051665 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:01.051671 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:01.051674 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:01.051905 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:01.052129 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:01.052130 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:01.052131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:01.052240 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:01.052515 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.052522 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.052661 (+ 139us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.052771 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.052929 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":105,"threads_started":1}
I20260927 18:44:01.053795 20343 heartbeater.cc:347] Connected to a master server at 127.15.110.254:34639
I20260927 18:44:01.053896 20343 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:01.054044 20343 heartbeater.cc:511] Master 127.15.110.254:34639 requested a full tablet report, sending...
I20260927 18:44:01.054481 19887 ts_manager.cc:194] Registered new tserver with Master: a02294a7f0d24b6991e5724dc5dcc67b (127.15.110.195:45291)
I20260927 18:44:01.054862 19887 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:41677
I20260927 18:44:01.059870 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:44:01.066782 19926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.061156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33892 (local address 127.15.110.254:34639)
0927 18:44:01.061225 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.061227 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.061318 (+ 91us) server_negotiation.cc:407] Connection header received
0927 18:44:01.061355 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.061356 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.061364 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.061411 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.061883 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.062375 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.063085 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.063319 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.063370 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.064615 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.064616 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.064616 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:01.064622 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.065807 (+ 1185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:01.066121 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:01.066122 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:01.066122 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:01.066169 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:01.066324 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:01.066324 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:01.066325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:01.066447 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.066525 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.066664 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.066723 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:44:01.068312 19887 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33892:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:01.070051 19887 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:01.076210 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58442 (local address 127.15.110.195:45291)
0927 18:44:01.072435 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.072440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.072451 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:44:01.072567 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.072570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.072589 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.072670 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:01.072935 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.073825 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.075138 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.075842 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.075901 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.075937 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.076118 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.076148 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":90,"threads_started":1}
I20260927 18:44:01.076246 20354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:45291 (local address 127.0.0.1:58442)
0927 18:44:01.072321 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.072359 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.072418 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.072762 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.072765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.072767 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:01.072831 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.072832 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.073921 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.073923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.075045 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.075047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.075919 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.075920 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.076003 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.076017 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.076101 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.076154 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":59,"threads_started":1}
I20260927 18:44:01.076520 20357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46163 (local address 127.0.0.1:45488)
0927 18:44:01.072788 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.072799 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.072849 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.073099 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.073100 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.073102 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:01.073143 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.073145 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.074391 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.074392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.075487 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.075489 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.076263 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.076264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.076340 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.076345 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.076391 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.076452 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"mutex_wait_us":11,"thread_start_us":60,"threads_started":1}
I20260927 18:44:01.076936 20277 tablet_service.cc:1467] Processing CreateTablet for tablet bc7c7d87e922459999f28170d2697c34 (DEFAULT_TABLE table=test-table [id=efcd11ce314749f09fb03c1c541cb92d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.077253 20277 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc7c7d87e922459999f28170d2697c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.077468 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45488 (local address 127.15.110.193:46163)
0927 18:44:01.072900 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.072904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.072914 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:01.072986 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.072989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.073005 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.073087 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:01.073202 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.074293 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.075574 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.076185 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.076254 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.076465 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.076492 (+ 27us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.076529 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260927 18:44:01.077497 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:34515 (local address 127.0.0.1:47720)
0927 18:44:01.072562 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.072572 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.072625 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.073328 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.073329 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.073331 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:01.073368 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.073369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.074576 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.074577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.075496 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.075498 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.077274 (+ 1776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.077275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.077336 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.077339 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.077390 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.077426 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260927 18:44:01.077957 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.072815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47720 (local address 127.15.110.194:34515)
0927 18:44:01.072973 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.072976 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.072985 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:01.073088 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.073090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.073108 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.073175 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:01.073462 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.074475 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.076651 (+ 2176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.077179 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.077273 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.077324 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.077825 (+ 501us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.077868 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":100,"threads_started":1}
I20260927 18:44:01.078075 20011 tablet_service.cc:1467] Processing CreateTablet for tablet bc7c7d87e922459999f28170d2697c34 (DEFAULT_TABLE table=test-table [id=efcd11ce314749f09fb03c1c541cb92d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.078367 20011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc7c7d87e922459999f28170d2697c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.078557 20144 tablet_service.cc:1467] Processing CreateTablet for tablet bc7c7d87e922459999f28170d2697c34 (DEFAULT_TABLE table=test-table [id=efcd11ce314749f09fb03c1c541cb92d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.078858 20144 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bc7c7d87e922459999f28170d2697c34. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.080615 20360 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Bootstrap starting.
I20260927 18:44:01.080924 20361 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Bootstrap starting.
I20260927 18:44:01.081403 20360 tablet_bootstrap.cc:654] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.081547 20361 tablet_bootstrap.cc:654] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.081985 20360 log.cc:824] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.082149 20361 log.cc:824] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.082222 20362 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Bootstrap starting.
I20260927 18:44:01.082811 20362 tablet_bootstrap.cc:654] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.082906 20360 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: No bootstrap required, opened a new log
I20260927 18:44:01.082993 20360 ts_tablet_manager.cc:1397] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:01.083033 20361 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: No bootstrap required, opened a new log
I20260927 18:44:01.083101 20361 ts_tablet_manager.cc:1397] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:01.083412 20362 log.cc:824] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.084198 20362 tablet_bootstrap.cc:492] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: No bootstrap required, opened a new log
I20260927 18:44:01.084264 20362 ts_tablet_manager.cc:1397] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:01.084786 20361 raft_consensus.cc:348] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.084933 20361 raft_consensus.cc:374] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.084976 20361 raft_consensus.cc:729] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfb49fd3d2c847cb868cde745e48028a, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.085105 20361 consensus_queue.cc:260] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.085206 20360 raft_consensus.cc:348] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.085371 20360 raft_consensus.cc:374] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.085384 20361 ts_tablet_manager.cc:1428] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:44:01.085413 20360 raft_consensus.cc:729] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a02294a7f0d24b6991e5724dc5dcc67b, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.085497 20077 heartbeater.cc:503] Master 127.15.110.254:34639 was elected leader, sending a full tablet report...
I20260927 18:44:01.085528 20360 consensus_queue.cc:260] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.085786 20360 ts_tablet_manager.cc:1428] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:01.085865 20343 heartbeater.cc:503] Master 127.15.110.254:34639 was elected leader, sending a full tablet report...
I20260927 18:44:01.086648 20362 raft_consensus.cc:348] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.086833 20362 raft_consensus.cc:374] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.086884 20362 raft_consensus.cc:729] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91ac88decf66405382aa3f05ba757772, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.086988 20362 consensus_queue.cc:260] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.087275 20362 ts_tablet_manager.cc:1428] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:01.087316 20210 heartbeater.cc:503] Master 127.15.110.254:34639 was elected leader, sending a full tablet report...
W20260927 18:44:01.137497 20211 tablet.cc:2369] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:01.219924 20078 tablet.cc:2369] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:01.293828 20344 tablet.cc:2369] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:01.321789 20367 raft_consensus.cc:482] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:01.321944 20367 raft_consensus.cc:504] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.322369 20367 leader_election.cc:288] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 91ac88decf66405382aa3f05ba757772 (127.15.110.194:34515), dfb49fd3d2c847cb868cde745e48028a (127.15.110.193:46163)
I20260927 18:44:01.325800 20359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.322805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:33013 (local address 127.15.110.194:34515)
0927 18:44:01.322869 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.322871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.322885 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:44:01.322920 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.322921 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.322931 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.323015 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:01.323301 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.323927 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325028 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.325425 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325482 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.325515 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.325679 (+ 164us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.325724 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:44:01.325825 20369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.322860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46163 (local address 127.15.110.195:34781)
0927 18:44:01.323065 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.323077 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.323141 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.323312 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.323312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.323314 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:01.323380 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.323381 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.324105 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.324105 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.324938 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.324941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.325548 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.325618 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.325621 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.325695 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.325733 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260927 18:44:01.325824 20358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.322968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:34781 (local address 127.15.110.193:46163)
0927 18:44:01.323038 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.323041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.323158 (+ 117us) server_negotiation.cc:407] Connection header received
0927 18:44:01.323227 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.323228 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.323238 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.323294 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:01.323450 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.324004 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325079 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.325431 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325532 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.325540 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.325716 (+ 176us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.325758 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:44:01.325824 20345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.322721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:34515 (local address 127.15.110.195:33013)
0927 18:44:01.322795 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.322809 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.322857 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.323054 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.323055 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.323057 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:01.323208 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.323210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.324041 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.324042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.324916 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:01.324918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.325513 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.325514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:01.325604 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.325607 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.325651 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.325709 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":64}
I20260927 18:44:01.326282 20164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc7c7d87e922459999f28170d2697c34" candidate_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ac88decf66405382aa3f05ba757772" is_pre_election: true
I20260927 18:44:01.326279 20031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc7c7d87e922459999f28170d2697c34" candidate_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfb49fd3d2c847cb868cde745e48028a" is_pre_election: true
I20260927 18:44:01.326443 20164 raft_consensus.cc:2454] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02294a7f0d24b6991e5724dc5dcc67b in term 0.
I20260927 18:44:01.326454 20031 raft_consensus.cc:2454] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a02294a7f0d24b6991e5724dc5dcc67b in term 0.
I20260927 18:44:01.326707 20230 leader_election.cc:302] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ac88decf66405382aa3f05ba757772, a02294a7f0d24b6991e5724dc5dcc67b; no voters:
I20260927 18:44:01.326851 20367 raft_consensus.cc:2781] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:01.326951 20367 raft_consensus.cc:482] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:01.326988 20367 raft_consensus.cc:3037] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:01.327838 20367 raft_consensus.cc:504] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.327991 20367 leader_election.cc:288] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [CANDIDATE]: Term 1 election: Requested vote from peers 91ac88decf66405382aa3f05ba757772 (127.15.110.194:34515), dfb49fd3d2c847cb868cde745e48028a (127.15.110.193:46163)
I20260927 18:44:01.328230 20031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc7c7d87e922459999f28170d2697c34" candidate_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfb49fd3d2c847cb868cde745e48028a"
I20260927 18:44:01.328230 20164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bc7c7d87e922459999f28170d2697c34" candidate_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "91ac88decf66405382aa3f05ba757772"
I20260927 18:44:01.328321 20164 raft_consensus.cc:3037] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:01.328321 20031 raft_consensus.cc:3037] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:01.329295 20031 raft_consensus.cc:2454] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02294a7f0d24b6991e5724dc5dcc67b in term 1.
I20260927 18:44:01.329308 20164 raft_consensus.cc:2454] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a02294a7f0d24b6991e5724dc5dcc67b in term 1.
I20260927 18:44:01.329500 20230 leader_election.cc:302] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 91ac88decf66405382aa3f05ba757772, a02294a7f0d24b6991e5724dc5dcc67b; no voters:
I20260927 18:44:01.329592 20367 raft_consensus.cc:2781] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:01.329775 20367 raft_consensus.cc:686] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 1 LEADER]: Becoming Leader. State: Replica: a02294a7f0d24b6991e5724dc5dcc67b, State: Running, Role: LEADER
I20260927 18:44:01.329886 20367 consensus_queue.cc:237] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } }
I20260927 18:44:01.330776 19887 catalog_manager.cc:5495] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b reported cstate change: term changed from 0 to 1, leader changed from <none> to a02294a7f0d24b6991e5724dc5dcc67b (127.15.110.195). New cstate: current_term: 1 leader_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a02294a7f0d24b6991e5724dc5dcc67b" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 45291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 } health_report { overall_health: UNKNOWN } } }
I20260927 18:44:01.360805 20355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.358113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.15.110.195:45291)
0927 18:44:01.358190 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.358192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.358202 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:01.358247 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.358249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.358271 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.358365 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:01.358523 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.359019 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.359906 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:01.360146 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:01.360223 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:01.360389 (+ 166us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:01.360519 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:01.360611 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.360662 (+ 51us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.360683 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:44:01.362927 20031 raft_consensus.cc:1261] T bc7c7d87e922459999f28170d2697c34 P dfb49fd3d2c847cb868cde745e48028a [term 1 FOLLOWER]: Refusing update from remote peer a02294a7f0d24b6991e5724dc5dcc67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:01.362942 20164 raft_consensus.cc:1261] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 1 FOLLOWER]: Refusing update from remote peer a02294a7f0d24b6991e5724dc5dcc67b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:01.363343 20367 consensus_queue.cc:1035] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfb49fd3d2c847cb868cde745e48028a" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:01.363504 20370 consensus_queue.cc:1035] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [LEADER]: Connected to new peer: Peer: permanent_uuid: "91ac88decf66405382aa3f05ba757772" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 34515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:01.366137 20373 mvcc.cc:205] Tried to move back new op lower bound from 7334029891019456512 to 7334029890887946240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029891019456512}]
I20260927 18:44:01.366446 20375 mvcc.cc:205] Tried to move back new op lower bound from 7334029891019456512 to 7334029890887946240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029891019456512}]
I20260927 18:44:01.366710 20374 mvcc.cc:205] Tried to move back new op lower bound from 7334029891019456512 to 7334029890887946240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029891019456512}]
I20260927 18:44:01.368700 19886 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:01.368806 19886 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33892:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:01.369597 19886 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0fa47574a8de4e5e94d9288eb4be7c44: Sending DeleteTablet for 3 replicas of tablet bc7c7d87e922459999f28170d2697c34
I20260927 18:44:01.369930 20277 tablet_service.cc:1514] Processing DeleteTablet for tablet bc7c7d87e922459999f28170d2697c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:58442
I20260927 18:44:01.369967 20011 tablet_service.cc:1514] Processing DeleteTablet for tablet bc7c7d87e922459999f28170d2697c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:45488
I20260927 18:44:01.370008 20144 tablet_service.cc:1514] Processing DeleteTablet for tablet bc7c7d87e922459999f28170d2697c34 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:01 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:47720
I20260927 18:44:01.370215 20383 tablet_replica.cc:330] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: stopping tablet replica
I20260927 18:44:01.370323 20383 raft_consensus.cc:2229] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:01.370450 20383 raft_consensus.cc:2258] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:01.370466 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19949
I20260927 18:44:01.371416 20383 ts_tablet_manager.cc:1905] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:01.373626 20383 ts_tablet_manager.cc:1918] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:01.373714 20383 log.cc:1196] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/bc7c7d87e922459999f28170d2697c34
I20260927 18:44:01.374002 20383 ts_tablet_manager.cc:1939] T bc7c7d87e922459999f28170d2697c34 P a02294a7f0d24b6991e5724dc5dcc67b: Deleting consensus metadata
I20260927 18:44:01.374557 19871 catalog_manager.cc:4841] TS a02294a7f0d24b6991e5724dc5dcc67b (127.15.110.195:45291): tablet bc7c7d87e922459999f28170d2697c34 (table test-table [id=efcd11ce314749f09fb03c1c541cb92d]) successfully deleted
W20260927 18:44:01.378309 19871 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:46163 (error 108)
I20260927 18:44:01.378443 20385 tablet_replica.cc:330] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772: stopping tablet replica
I20260927 18:44:01.378589 20385 raft_consensus.cc:2229] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:01.378727 20385 raft_consensus.cc:2258] T bc7c7d87e922459999f28170d2697c34 P 91ac88decf66405382aa3f05ba757772 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:01.379053 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.378879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46163 (local address 127.0.0.1:45498)
0927 18:44:01.378953 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.379017 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46163: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:44:01.379205 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20082
W20260927 18:44:01.379211 19871 catalog_manager.cc:4584] TS dfb49fd3d2c847cb868cde745e48028a (127.15.110.193:46163): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bc7c7d87e922459999f28170d2697c34: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46163: connect: Connection refused (error 111)
I20260927 18:44:01.383916 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20215
I20260927 18:44:01.384042 20356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.383906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:34515 (local address 127.0.0.1:47732)
0927 18:44:01.383965 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.384001 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.194:34515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:44:01.388965 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 19856
2026-09-27T18:44:01Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2160 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-27T18:44:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:01Z Disabled control of system clock
I20260927 18:44:01.417425 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33539
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:33539
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:44:01.498271 20390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:01.498484 20390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:01.498505 20390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:01.499631 20390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:01.499667 20390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:01.499682 20390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:01.499694 20390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:01.499706 20390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:01.500977 20390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:33539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:01.501178 20390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:01.501494 20390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:01.506484 20396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.506513 20398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.506541 20390 server_base.cc:1034] running on GCE node
W20260927 18:44:01.506513 20395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.506964 20390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:01.507288 20390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:01.508430 20390 hybrid_clock.cc:648] HybridClock initialized: now 1790534641508411 us; error 38 us; skew 500 ppm
I20260927 18:44:01.509574 20390 webserver.cc:460] Webserver started at http://127.15.110.254:43731/ using document root <none> and password file <none>
I20260927 18:44:01.509829 20390 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:01.509877 20390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:01.510035 20390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:01.511142 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "e12746a32fda4849b1de20ccfeef6468"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "914b34d7ae996fe9928183297c3432db"
server_key_iv: "75330bdf3cbceeaa34ef00f58d39683a"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.511587 20390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "e12746a32fda4849b1de20ccfeef6468"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "914b34d7ae996fe9928183297c3432db"
server_key_iv: "75330bdf3cbceeaa34ef00f58d39683a"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.513010 20390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:01.513964 20404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.514221 20390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:01.514307 20390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "e12746a32fda4849b1de20ccfeef6468"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "914b34d7ae996fe9928183297c3432db"
server_key_iv: "75330bdf3cbceeaa34ef00f58d39683a"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.514407 20390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:01.517915 20390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:01.518209 20390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:01.518353 20390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:01.522426 20390 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:33539
I20260927 18:44:01.522447 20456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:33539 every 8 connection(s)
I20260927 18:44:01.522789 20390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:01.523491 20457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.526156 20457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: Bootstrap starting.
I20260927 18:44:01.526729 20457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.527010 20457 log.cc:824] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.527729 20457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: No bootstrap required, opened a new log
I20260927 18:44:01.529279 20457 raft_consensus.cc:348] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } }
I20260927 18:44:01.529440 20457 raft_consensus.cc:374] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.529492 20457 raft_consensus.cc:729] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e12746a32fda4849b1de20ccfeef6468, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.529673 20457 consensus_queue.cc:260] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } }
I20260927 18:44:01.529772 20457 raft_consensus.cc:388] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:01.529816 20457 raft_consensus.cc:482] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:01.529865 20457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:01.530403 20457 raft_consensus.cc:504] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } }
I20260927 18:44:01.530520 20457 leader_election.cc:302] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e12746a32fda4849b1de20ccfeef6468; no voters:
I20260927 18:44:01.530720 20457 leader_election.cc:288] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:01.530836 20460 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:01.531032 20460 raft_consensus.cc:686] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [term 1 LEADER]: Becoming Leader. State: Replica: e12746a32fda4849b1de20ccfeef6468, State: Running, Role: LEADER
I20260927 18:44:01.531035 20457 sys_catalog.cc:564] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:01.531131 20460 consensus_queue.cc:237] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } }
I20260927 18:44:01.531721 20461 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e12746a32fda4849b1de20ccfeef6468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } } }
I20260927 18:44:01.531808 20461 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:01.531910 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e12746a32fda4849b1de20ccfeef6468. Latest consensus state: current_term: 1 leader_uuid: "e12746a32fda4849b1de20ccfeef6468" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e12746a32fda4849b1de20ccfeef6468" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33539 } } }
I20260927 18:44:01.531980 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:01.532248 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20390
I20260927 18:44:01.532347 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:01.532557 15803 external_mini_cluster.cc:1428] Setting key bb611efd84b345c3b8aba903561e18f1
I20260927 18:44:01.532567 20469 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:01.533072 20469 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:01.534659 20469 catalog_manager.cc:1347] Generated new cluster ID: e9c30f615f9f40a18f775cc10714b885
I20260927 18:44:01.534715 20469 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:01.544961 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.533057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56656 (local address 127.15.110.254:33539)
0927 18:44:01.543322 (+ 10265us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.543330 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.543364 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:44:01.543467 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.543478 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.543520 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.543655 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.544466 (+ 811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.544470 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.544470 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.544475 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.544528 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.544540 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.544617 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.544691 (+ 74us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.544755 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10192,"thread_start_us":68,"threads_started":1}
I20260927 18:44:01.551618 20469 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:01.551790 20469 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:01.558015 20469 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: Generated new TSK 0
I20260927 18:44:01.558198 20469 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:01.563373 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--builtin_ntp_servers=127.15.110.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:44:01.644425 20481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:01.644649 20481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:01.644673 20481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:01.645884 20481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:01.645924 20481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:01.645951 20481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:01.647629 20481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:01.647927 20481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:01.648286 20481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:01.654289 20487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.654289 20489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.654531 20481 server_base.cc:1034] running on GCE node
W20260927 18:44:01.654295 20486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.654899 20481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:01.655157 20481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:01.656349 20481 hybrid_clock.cc:648] HybridClock initialized: now 1790534641656335 us; error 26 us; skew 500 ppm
I20260927 18:44:01.657945 20481 webserver.cc:460] Webserver started at http://127.15.110.193:42367/ using document root <none> and password file <none>
I20260927 18:44:01.658241 20481 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:01.658313 20481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:01.658443 20481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:01.659866 20481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "225e7a463ddf46e8809e1c3187371fcc"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "3f2d68f3bb24699aa80f23215d3c1434"
server_key_iv: "bd7021512ae66aed26aa85ad995deb5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.660265 20481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "225e7a463ddf46e8809e1c3187371fcc"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "3f2d68f3bb24699aa80f23215d3c1434"
server_key_iv: "bd7021512ae66aed26aa85ad995deb5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.661811 20481 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:44:01.662726 20495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.662982 20481 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:01.663069 20481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "225e7a463ddf46e8809e1c3187371fcc"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "3f2d68f3bb24699aa80f23215d3c1434"
server_key_iv: "bd7021512ae66aed26aa85ad995deb5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.663136 20481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:01.667881 20481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:01.668183 20481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:01.668323 20481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:01.668552 20481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:01.668838 20481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:01.668869 20481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.668890 20481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:01.668903 20481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.674757 20481 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:45533
I20260927 18:44:01.674803 20608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:45533 every 8 connection(s)
I20260927 18:44:01.675122 20481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:01.678716 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.675942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:53781 (local address 127.15.110.254:33539)
0927 18:44:01.675987 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.675989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.676238 (+ 249us) server_negotiation.cc:407] Connection header received
0927 18:44:01.676757 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.676758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.676766 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.676849 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.678374 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.678376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.678376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.678381 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.678400 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.678406 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.678474 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.678603 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.678617 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:01.679026 20611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.675888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33539 (local address 127.15.110.193:53781)
0927 18:44:01.676102 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.676141 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.676617 (+ 476us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.676924 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.676928 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.676936 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.678156 (+ 1220us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.678156 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.678228 (+ 72us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.678270 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.678497 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.678503 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.678506 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.678608 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.678674 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":98,"threads_started":1}
I20260927 18:44:01.679106 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20481
I20260927 18:44:01.679241 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:01.679455 15803 external_mini_cluster.cc:1428] Setting key 150742d9910e43b08225090b77163e1e
I20260927 18:44:01.680585 20609 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33539
I20260927 18:44:01.680693 20609 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:01.680845 20609 heartbeater.cc:511] Master 127.15.110.254:33539 requested a full tablet report, sending...
I20260927 18:44:01.680832 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--builtin_ntp_servers=127.15.110.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260927 18:44:01.681402 20421 ts_manager.cc:194] Registered new tserver with Master: 225e7a463ddf46e8809e1c3187371fcc (127.15.110.193:45533)
W20260927 18:44:01.761055 20612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:01.761279 20612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:01.761303 20612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:01.762703 20612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:01.762781 20612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:01.762816 20612 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:01.764606 20612 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:01.764894 20612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:01.765292 20612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:01.771104 20618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.771085 20612 server_base.cc:1034] running on GCE node
W20260927 18:44:01.771163 20620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.771090 20617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.771557 20612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:01.771839 20612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:01.773011 20612 hybrid_clock.cc:648] HybridClock initialized: now 1790534641773000 us; error 33 us; skew 500 ppm
I20260927 18:44:01.774312 20612 webserver.cc:460] Webserver started at http://127.15.110.194:40171/ using document root <none> and password file <none>
I20260927 18:44:01.774569 20612 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:01.774652 20612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:01.774801 20612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:01.775977 20612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "4eac399d357f4cb2a64a101b4a8e7ea7"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "d7d1a926fc5edbd96877061513b0d75c"
server_key_iv: "15b630b4e7dfa7c799cf9f3d4f2367a5"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.776367 20612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "4eac399d357f4cb2a64a101b4a8e7ea7"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "d7d1a926fc5edbd96877061513b0d75c"
server_key_iv: "15b630b4e7dfa7c799cf9f3d4f2367a5"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.777765 20612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:01.778671 20626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.778848 20612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:01.778957 20612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "4eac399d357f4cb2a64a101b4a8e7ea7"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "d7d1a926fc5edbd96877061513b0d75c"
server_key_iv: "15b630b4e7dfa7c799cf9f3d4f2367a5"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.779047 20612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:01.782588 20612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:01.782863 20612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:01.783007 20612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:01.783253 20612 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:01.783584 20612 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:01.783639 20612 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.783677 20612 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:01.783704 20612 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.789636 20612 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:44409
I20260927 18:44:01.789678 20739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:44409 every 8 connection(s)
I20260927 18:44:01.790027 20612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:01.793126 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.790727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54103 (local address 127.15.110.254:33539)
0927 18:44:01.790775 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.790777 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.791037 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:44:01.791608 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.791610 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.791615 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.791675 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.792822 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.792823 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.792824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.792831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.792845 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.792849 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.792898 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.793056 (+ 158us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.793070 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:44:01.793282 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.790718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33539 (local address 127.15.110.194:54103)
0927 18:44:01.790901 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.790941 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.791485 (+ 544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.791731 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.791734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.791743 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.792672 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.792673 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.792701 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.792730 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.792940 (+ 210us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.792946 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.792949 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.793031 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.793084 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260927 18:44:01.793798 20740 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33539
I20260927 18:44:01.793916 20740 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:01.794093 20740 heartbeater.cc:511] Master 127.15.110.254:33539 requested a full tablet report, sending...
I20260927 18:44:01.794450 20421 ts_manager.cc:194] Registered new tserver with Master: 4eac399d357f4cb2a64a101b4a8e7ea7 (127.15.110.194:44409)
I20260927 18:44:01.795444 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20612
I20260927 18:44:01.795600 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:01.795800 15803 external_mini_cluster.cc:1428] Setting key fdfb830cd674f1f3425d2c3f399afd76
I20260927 18:44:01.796792 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--builtin_ntp_servers=127.15.110.212:37179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260927 18:44:01.879462 20743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:01.879680 20743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:01.879702 20743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:01.880807 20743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:01.880841 20743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:01.880867 20743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:01.882288 20743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33539
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:01.882496 20743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:01.882740 20743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:01.887773 20748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.887748 20749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:01.887746 20751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:01.887956 20743 server_base.cc:1034] running on GCE node
I20260927 18:44:01.888236 20743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:01.888484 20743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:01.889648 20743 hybrid_clock.cc:648] HybridClock initialized: now 1790534641889641 us; error 33 us; skew 500 ppm
I20260927 18:44:01.890939 20743 webserver.cc:460] Webserver started at http://127.15.110.195:36553/ using document root <none> and password file <none>
I20260927 18:44:01.891242 20743 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:01.891317 20743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:01.891458 20743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:01.892549 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "b1d99afc58274166950abf35256d3ca3"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "7ef501cdb320e9ae5fb4d9ffb8c9b8f4"
server_key_iv: "8462a03a2e1871204e2ba8019c3d9f5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.892916 20743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "b1d99afc58274166950abf35256d3ca3"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "7ef501cdb320e9ae5fb4d9ffb8c9b8f4"
server_key_iv: "8462a03a2e1871204e2ba8019c3d9f5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.894227 20743 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:01.894941 20757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.895125 20743 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:01.895253 20743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "b1d99afc58274166950abf35256d3ca3"
format_stamp: "Formatted at 2026-09-27 18:44:01 on dist-test-slave-vs05"
server_key: "7ef501cdb320e9ae5fb4d9ffb8c9b8f4"
server_key_iv: "8462a03a2e1871204e2ba8019c3d9f5c"
server_key_version: "encryptionkey@0"
I20260927 18:44:01.895339 20743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:01.898635 20743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:01.898890 20743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:01.899031 20743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:01.899293 20743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:01.899591 20743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:01.899644 20743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.899682 20743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:01.899709 20743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:01.905632 20743 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:34301
I20260927 18:44:01.905786 20870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:34301 every 8 connection(s)
I20260927 18:44:01.906077 20743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:01.909055 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.906839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:48307 (local address 127.15.110.254:33539)
0927 18:44:01.906889 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.906892 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.907238 (+ 346us) server_negotiation.cc:407] Connection header received
0927 18:44:01.907745 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.907747 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.907752 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.907804 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.908845 (+ 1041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.908847 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.908847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.908852 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.908869 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.908874 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.908941 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.908989 (+ 48us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.909002 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:44:01.909299 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.906831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33539 (local address 127.15.110.195:48307)
0927 18:44:01.907034 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.907113 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.907655 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.907858 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.907861 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.907868 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.908697 (+ 829us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.908697 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.908724 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.908753 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.908940 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.908943 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.908945 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.908994 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.909039 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":100,"threads_started":1}
I20260927 18:44:01.909772 20871 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33539
I20260927 18:44:01.909860 20871 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:01.910012 20871 heartbeater.cc:511] Master 127.15.110.254:33539 requested a full tablet report, sending...
I20260927 18:44:01.910444 20421 ts_manager.cc:194] Registered new tserver with Master: b1d99afc58274166950abf35256d3ca3 (127.15.110.195:34301)
I20260927 18:44:01.911784 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20743
I20260927 18:44:01.911870 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:01.912004 15803 external_mini_cluster.cc:1428] Setting key 54df2be7990ac384759ef3d592e392de
I20260927 18:44:01.912897 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:44:01.915014 20479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.914159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56670 (local address 127.15.110.254:33539)
0927 18:44:01.914200 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.914201 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.914322 (+ 121us) server_negotiation.cc:407] Connection header received
0927 18:44:01.914352 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.914352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.914357 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.914402 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.914863 (+ 461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.914864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.914865 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.914868 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.914881 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.914884 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.914924 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.914956 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.914968 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:01.916308 20421 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56670:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:01.917917 20421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:01.921895 20881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:45533 (local address 127.0.0.1:53460)
0927 18:44:01.920492 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.920528 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.920624 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.920844 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.920850 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.920898 (+ 48us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.921387 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.921387 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.921405 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.921432 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.921705 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.921711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.921714 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.921776 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.921795 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":83,"threads_started":1}
I20260927 18:44:01.922155 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53460 (local address 127.15.110.193:45533)
0927 18:44:01.920466 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.920470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.920613 (+ 143us) server_negotiation.cc:407] Connection header received
0927 18:44:01.920690 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.920693 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.920704 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.920788 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.921519 (+ 731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.921528 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.921528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.921533 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.921587 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.921616 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.921676 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.922046 (+ 370us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.922063 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":74,"threads_started":1}
I20260927 18:44:01.922260 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44409 (local address 127.0.0.1:52476)
0927 18:44:01.920273 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.920709 (+ 436us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.920755 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.921482 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.921483 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.921485 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.921872 (+ 387us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.921873 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.921878 (+ 5us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.921888 (+ 10us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.922147 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.922149 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.922149 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.922195 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.922206 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":54,"threads_started":1}
I20260927 18:44:01.922329 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.15.110.195:34301)
0927 18:44:01.920466 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.920470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.920925 (+ 455us) server_negotiation.cc:407] Connection header received
0927 18:44:01.920954 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.920956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.920966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.921030 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.921773 (+ 743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.921781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.921781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.921786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.921819 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.921843 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.921879 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.922169 (+ 290us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.922186 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260927 18:44:01.922101 20879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:34301 (local address 127.0.0.1:50598)
0927 18:44:01.920826 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:01.920834 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:01.920869 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:01.921265 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:01.921266 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:01.921269 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:01.921665 (+ 396us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:01.921665 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:01.921672 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:01.921684 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:01.921954 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:01.921955 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:01.921955 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 18:44:01.922002 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:44:01.922015 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":87,"threads_started":1}
I20260927 18:44:01.922595 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:01.920189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52476 (local address 127.15.110.194:44409)
0927 18:44:01.920379 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:01.920385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:01.921068 (+ 683us) server_negotiation.cc:407] Connection header received
0927 18:44:01.921387 (+ 319us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:01.921391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:01.921401 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:01.921484 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:01.921977 (+ 493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:01.921986 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:01.921987 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:01.921991 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:01.922024 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:01.922052 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:01.922120 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:01.922459 (+ 339us) server_negotiation.cc:299] Negotiation successful
0927 18:44:01.922478 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
I20260927 18:44:01.923132 20805 tablet_service.cc:1467] Processing CreateTablet for tablet e30abf2622b340b480acc467ac9e2058 (DEFAULT_TABLE table=test-table [id=2e8385835e1d4beea58c81c9c119a2c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.923138 20543 tablet_service.cc:1467] Processing CreateTablet for tablet e30abf2622b340b480acc467ac9e2058 (DEFAULT_TABLE table=test-table [id=2e8385835e1d4beea58c81c9c119a2c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.923537 20543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30abf2622b340b480acc467ac9e2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.923537 20805 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30abf2622b340b480acc467ac9e2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.926942 20886 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: Bootstrap starting.
I20260927 18:44:01.927379 20674 tablet_service.cc:1467] Processing CreateTablet for tablet e30abf2622b340b480acc467ac9e2058 (DEFAULT_TABLE table=test-table [id=2e8385835e1d4beea58c81c9c119a2c7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:01.927724 20886 tablet_bootstrap.cc:654] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.927719 20674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e30abf2622b340b480acc467ac9e2058. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:01.928156 20886 log.cc:824] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.928532 20885 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Bootstrap starting.
I20260927 18:44:01.929061 20885 tablet_bootstrap.cc:654] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.929352 20885 log.cc:824] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.929922 20886 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: No bootstrap required, opened a new log
I20260927 18:44:01.930006 20886 ts_tablet_manager.cc:1397] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:44:01.930114 20885 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: No bootstrap required, opened a new log
I20260927 18:44:01.930191 20885 ts_tablet_manager.cc:1397] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:44:01.930812 20889 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Bootstrap starting.
I20260927 18:44:01.931666 20889 tablet_bootstrap.cc:654] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:01.932106 20889 log.cc:824] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:01.932857 20886 raft_consensus.cc:348] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.933012 20889 tablet_bootstrap.cc:492] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: No bootstrap required, opened a new log
I20260927 18:44:01.933029 20886 raft_consensus.cc:374] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.932936 20885 raft_consensus.cc:348] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.933068 20889 ts_tablet_manager.cc:1397] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:44:01.933066 20886 raft_consensus.cc:729] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 225e7a463ddf46e8809e1c3187371fcc, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.933092 20885 raft_consensus.cc:374] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.933126 20885 raft_consensus.cc:729] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1d99afc58274166950abf35256d3ca3, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.933229 20886 consensus_queue.cc:260] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.933254 20885 consensus_queue.cc:260] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.933497 20886 ts_tablet_manager.cc:1428] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:44:01.933530 20885 ts_tablet_manager.cc:1428] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:44:01.933549 20871 heartbeater.cc:503] Master 127.15.110.254:33539 was elected leader, sending a full tablet report...
I20260927 18:44:01.933609 20609 heartbeater.cc:503] Master 127.15.110.254:33539 was elected leader, sending a full tablet report...
I20260927 18:44:01.935551 20889 raft_consensus.cc:348] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.935676 20889 raft_consensus.cc:374] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:01.935727 20889 raft_consensus.cc:729] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eac399d357f4cb2a64a101b4a8e7ea7, State: Initialized, Role: FOLLOWER
I20260927 18:44:01.935842 20889 consensus_queue.cc:260] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:01.936106 20889 ts_tablet_manager.cc:1428] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:44:01.936158 20740 heartbeater.cc:503] Master 127.15.110.254:33539 was elected leader, sending a full tablet report...
W20260927 18:44:02.040609 20741 tablet.cc:2369] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:02.123162 20892 raft_consensus.cc:482] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:02.123350 20892 raft_consensus.cc:504] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:02.123683 20892 leader_election.cc:288] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4eac399d357f4cb2a64a101b4a8e7ea7 (127.15.110.194:44409), 225e7a463ddf46e8809e1c3187371fcc (127.15.110.193:45533)
I20260927 18:44:02.125160 20873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.123912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:45533 (local address 127.15.110.195:36349)
0927 18:44:02.124008 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.124023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:02.124087 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:02.124248 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:02.124248 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:02.124251 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:02.124761 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:02.124761 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:02.124769 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:02.124784 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:02.124972 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:02.124973 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:02.124974 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:44:02.125060 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:44:02.125075 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260927 18:44:02.125217 20884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.124010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:36349 (local address 127.15.110.193:45533)
0927 18:44:02.124080 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.124083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.124095 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:44:02.124154 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.124156 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.124161 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.124243 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.124881 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.124882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.124882 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.124887 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.124907 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.124912 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.124976 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.125094 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.125123 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:44:02.125289 20894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.123912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44409 (local address 127.15.110.195:49673)
0927 18:44:02.124224 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.124232 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:02.124305 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:02.124430 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:02.124431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:02.124444 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:02.124933 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:02.124934 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:02.124941 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:02.124954 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:02.125138 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:02.125139 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:02.125139 (+ 0us) client_negotiation.cc:769] Sending connection context
0927 18:44:02.125192 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:44:02.125205 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"mutex_wait_us":146,"thread_start_us":89,"threads_started":1}
I20260927 18:44:02.125293 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.124015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49673 (local address 127.15.110.194:44409)
0927 18:44:02.124089 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.124092 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.124296 (+ 204us) server_negotiation.cc:407] Connection header received
0927 18:44:02.124361 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.124362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.124367 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.124425 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.125041 (+ 616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.125043 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.125044 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.125049 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.125069 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.125074 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.125131 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.125193 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.125208 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:44:02.125558 20563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30abf2622b340b480acc467ac9e2058" candidate_uuid: "b1d99afc58274166950abf35256d3ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225e7a463ddf46e8809e1c3187371fcc" is_pre_election: true
I20260927 18:44:02.125759 20563 raft_consensus.cc:2454] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d99afc58274166950abf35256d3ca3 in term 0.
I20260927 18:44:02.125708 20694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30abf2622b340b480acc467ac9e2058" candidate_uuid: "b1d99afc58274166950abf35256d3ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" is_pre_election: true
I20260927 18:44:02.125873 20694 raft_consensus.cc:2454] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b1d99afc58274166950abf35256d3ca3 in term 0.
I20260927 18:44:02.126020 20759 leader_election.cc:302] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225e7a463ddf46e8809e1c3187371fcc, b1d99afc58274166950abf35256d3ca3; no voters:
I20260927 18:44:02.126153 20892 raft_consensus.cc:2781] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:02.126238 20892 raft_consensus.cc:482] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:02.126271 20892 raft_consensus.cc:3037] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.126982 20892 raft_consensus.cc:504] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:02.127138 20892 leader_election.cc:288] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [CANDIDATE]: Term 1 election: Requested vote from peers 4eac399d357f4cb2a64a101b4a8e7ea7 (127.15.110.194:44409), 225e7a463ddf46e8809e1c3187371fcc (127.15.110.193:45533)
I20260927 18:44:02.127418 20694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30abf2622b340b480acc467ac9e2058" candidate_uuid: "b1d99afc58274166950abf35256d3ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7"
I20260927 18:44:02.127339 20563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e30abf2622b340b480acc467ac9e2058" candidate_uuid: "b1d99afc58274166950abf35256d3ca3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "225e7a463ddf46e8809e1c3187371fcc"
I20260927 18:44:02.127496 20694 raft_consensus.cc:3037] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.127504 20563 raft_consensus.cc:3037] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.128428 20563 raft_consensus.cc:2454] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d99afc58274166950abf35256d3ca3 in term 1.
I20260927 18:44:02.128422 20694 raft_consensus.cc:2454] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b1d99afc58274166950abf35256d3ca3 in term 1.
I20260927 18:44:02.128598 20759 leader_election.cc:302] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 225e7a463ddf46e8809e1c3187371fcc, b1d99afc58274166950abf35256d3ca3; no voters:
I20260927 18:44:02.128711 20892 raft_consensus.cc:2781] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:02.128885 20892 raft_consensus.cc:686] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 1 LEADER]: Becoming Leader. State: Replica: b1d99afc58274166950abf35256d3ca3, State: Running, Role: LEADER
I20260927 18:44:02.129010 20892 consensus_queue.cc:237] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } }
I20260927 18:44:02.129858 20420 catalog_manager.cc:5495] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b1d99afc58274166950abf35256d3ca3 (127.15.110.195). New cstate: current_term: 1 leader_uuid: "b1d99afc58274166950abf35256d3ca3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1d99afc58274166950abf35256d3ca3" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 34301 } health_report { overall_health: HEALTHY } } }
I20260927 18:44:02.148561 20883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.147669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50600 (local address 127.15.110.195:34301)
0927 18:44:02.147737 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.147738 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.147746 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:02.147779 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.147780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.147792 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.147835 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.148409 (+ 574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.148410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.148410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.148412 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.148423 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.148426 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.148466 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.148498 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.148508 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:44:02.150389 20694 raft_consensus.cc:1261] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 1 FOLLOWER]: Refusing update from remote peer b1d99afc58274166950abf35256d3ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:02.150389 20563 raft_consensus.cc:1261] T e30abf2622b340b480acc467ac9e2058 P 225e7a463ddf46e8809e1c3187371fcc [term 1 FOLLOWER]: Refusing update from remote peer b1d99afc58274166950abf35256d3ca3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:02.150730 20895 consensus_queue.cc:1035] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "225e7a463ddf46e8809e1c3187371fcc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 45533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:02.150864 20892 consensus_queue.cc:1035] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eac399d357f4cb2a64a101b4a8e7ea7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:02.153290 20898 mvcc.cc:205] Tried to move back new op lower bound from 7334029894245203968 to 7334029894161182720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029894245203968}]
I20260927 18:44:02.153421 20899 mvcc.cc:205] Tried to move back new op lower bound from 7334029894245203968 to 7334029894161182720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029894245203968}]
I20260927 18:44:02.154114 20900 mvcc.cc:205] Tried to move back new op lower bound from 7334029894245203968 to 7334029894161182720. Current Snapshot: MvccSnapshot[applied={T|T < 7334029894245203968 or (T in {7334029894245203968})}]
I20260927 18:44:02.155937 20421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:02.156024 20421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56670:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:02.156821 20421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e12746a32fda4849b1de20ccfeef6468: Sending DeleteTablet for 3 replicas of tablet e30abf2622b340b480acc467ac9e2058
I20260927 18:44:02.157119 20674 tablet_service.cc:1514] Processing DeleteTablet for tablet e30abf2622b340b480acc467ac9e2058 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:02 UTC) from {username='slave'} at 127.0.0.1:52476
I20260927 18:44:02.157151 20805 tablet_service.cc:1514] Processing DeleteTablet for tablet e30abf2622b340b480acc467ac9e2058 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:02 UTC) from {username='slave'} at 127.0.0.1:50598
I20260927 18:44:02.157353 20909 tablet_replica.cc:330] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: stopping tablet replica
I20260927 18:44:02.157387 20908 tablet_replica.cc:330] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: stopping tablet replica
I20260927 18:44:02.157359 20543 tablet_service.cc:1514] Processing DeleteTablet for tablet e30abf2622b340b480acc467ac9e2058 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:02 UTC) from {username='slave'} at 127.0.0.1:53460
I20260927 18:44:02.157464 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20481
I20260927 18:44:02.157480 20908 raft_consensus.cc:2229] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:02.157506 20909 raft_consensus.cc:2229] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:02.158195 20909 raft_consensus.cc:2258] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:02.158649 20909 ts_tablet_manager.cc:1905] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:02.158720 20908 raft_consensus.cc:2258] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:02.159199 20908 ts_tablet_manager.cc:1905] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:02.160518 20909 ts_tablet_manager.cc:1918] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:02.160549 20908 ts_tablet_manager.cc:1918] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:02.160596 20909 log.cc:1196] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/e30abf2622b340b480acc467ac9e2058
I20260927 18:44:02.160633 20908 log.cc:1196] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/e30abf2622b340b480acc467ac9e2058
I20260927 18:44:02.160872 20908 ts_tablet_manager.cc:1939] T e30abf2622b340b480acc467ac9e2058 P 4eac399d357f4cb2a64a101b4a8e7ea7: Deleting consensus metadata
I20260927 18:44:02.160868 20909 ts_tablet_manager.cc:1939] T e30abf2622b340b480acc467ac9e2058 P b1d99afc58274166950abf35256d3ca3: Deleting consensus metadata
I20260927 18:44:02.161273 20405 catalog_manager.cc:4841] TS b1d99afc58274166950abf35256d3ca3 (127.15.110.195:34301): tablet e30abf2622b340b480acc467ac9e2058 (table test-table [id=2e8385835e1d4beea58c81c9c119a2c7]) successfully deleted
I20260927 18:44:02.161283 20407 catalog_manager.cc:4841] TS 4eac399d357f4cb2a64a101b4a8e7ea7 (127.15.110.194:44409): tablet e30abf2622b340b480acc467ac9e2058 (table test-table [id=2e8385835e1d4beea58c81c9c119a2c7]) successfully deleted
W20260927 18:44:02.166388 20406 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:45533 (error 108)
I20260927 18:44:02.166647 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20612
I20260927 18:44:02.167224 20880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.166765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:45533 (local address 127.0.0.1:53466)
0927 18:44:02.167123 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.167166 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:45533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336}
W20260927 18:44:02.167586 20406 catalog_manager.cc:4584] TS 225e7a463ddf46e8809e1c3187371fcc (127.15.110.193:45533): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e30abf2622b340b480acc467ac9e2058: Network error: Client connection negotiation failed: client connection to 127.15.110.193:45533: connect: Connection refused (error 111)
I20260927 18:44:02.171427 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20743
I20260927 18:44:02.176012 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20390
2026-09-27T18:44:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (784 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260927 18:44:02.194772 15803 mini_oidc.cc:152] Starting JWKS server
I20260927 18:44:02.195896 15803 webserver.cc:460] Webserver started at https://127.0.0.1:37597/ using document root <none> and password file <none>
I20260927 18:44:02.196027 15803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:44:02.196233 15803 webserver.cc:460] Webserver started at http://127.0.0.1:41085/ using document root <none> and password file <none>
2026-09-27T18:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:02Z Disabled control of system clock
I20260927 18:44:02.204424 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33377
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:39111
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:33377
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/testchainca.cert with env {}
W20260927 18:44:02.285126 20917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:02.285344 20917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:02.285367 20917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:02.286484 20917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:02.286521 20917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:02.286533 20917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:02.286546 20917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:02.286558 20917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:44:02.286571 20917 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:44:02.286581 20917 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:37597/jwks.json
I20260927 18:44:02.288101 20917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39111
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:33377
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:37597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:02.288376 20917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:02.288565 20917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:02.293466 20922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.293476 20923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.293512 20925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.293779 20917 server_base.cc:1034] running on GCE node
I20260927 18:44:02.294011 20917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:02.294312 20917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:02.295483 20917 hybrid_clock.cc:648] HybridClock initialized: now 1790534642295495 us; error 71 us; skew 500 ppm
I20260927 18:44:02.296730 20917 webserver.cc:460] Webserver started at http://127.15.110.254:42347/ using document root <none> and password file <none>
I20260927 18:44:02.296993 20917 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:02.297065 20917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:02.297197 20917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:02.298245 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "7009da46005b70ddd13e6b1d0d4cffd9"
server_key_iv: "41142e193ae6d1a1e716de58803c1225"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.298628 20917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "7009da46005b70ddd13e6b1d0d4cffd9"
server_key_iv: "41142e193ae6d1a1e716de58803c1225"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.299947 20917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:02.300683 20931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:02.300877 20917 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:02.300948 20917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "7009da46005b70ddd13e6b1d0d4cffd9"
server_key_iv: "41142e193ae6d1a1e716de58803c1225"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.301000 20917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:02.322147 20917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:02.322458 20917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:02.322576 20917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:02.326838 20917 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:33377
I20260927 18:44:02.327026 20983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:33377 every 8 connection(s)
I20260927 18:44:02.327327 20917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:02.328372 20984 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:02.328994 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 20917
I20260927 18:44:02.329097 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:02.329329 15803 external_mini_cluster.cc:1428] Setting key 5a23f06c2a715af7fb1441372766d5f3
I20260927 18:44:02.331422 20984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd: Bootstrap starting.
I20260927 18:44:02.332064 20984 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:02.332379 20984 log.cc:824] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:02.333132 20984 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd: No bootstrap required, opened a new log
I20260927 18:44:02.334874 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.329906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33942 (local address 127.15.110.254:33377)
0927 18:44:02.330097 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.330104 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.330130 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:44:02.330523 (+ 393us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.330529 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.330599 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.330716 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.330947 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.331907 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.332996 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.333444 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.333539 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.334191 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.334195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.334195 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.334361 (+ 166us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.334393 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.334403 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.334452 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.334594 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.334688 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260927 18:44:02.335536 20984 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } }
I20260927 18:44:02.335711 20984 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:02.335757 20984 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2ca21cf24046aeb6bcacf7017bbdcd, State: Initialized, Role: FOLLOWER
I20260927 18:44:02.335868 20984 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } }
I20260927 18:44:02.335951 20984 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:02.335943 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:44:02.335983 20984 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:02.336038 20984 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.336720 20984 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } }
I20260927 18:44:02.336838 20984 leader_election.cc:302] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d2ca21cf24046aeb6bcacf7017bbdcd; no voters:
I20260927 18:44:02.337026 20984 leader_election.cc:288] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:02.337035 20989 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:02.337208 20989 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [term 1 LEADER]: Becoming Leader. State: Replica: 7d2ca21cf24046aeb6bcacf7017bbdcd, State: Running, Role: LEADER
I20260927 18:44:02.337297 20989 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } }
I20260927 18:44:02.337337 20984 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:02.337610 20991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d2ca21cf24046aeb6bcacf7017bbdcd. Latest consensus state: current_term: 1 leader_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } } }
I20260927 18:44:02.337615 20990 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d2ca21cf24046aeb6bcacf7017bbdcd" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33377 } } }
I20260927 18:44:02.337708 20991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.337715 20990 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.337947 20995 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:02.338461 20995 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:02.340080 20995 catalog_manager.cc:1347] Generated new cluster ID: 841526adcd3c43c68878a35ce87c688d
I20260927 18:44:02.340132 20995 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:02.362102 20995 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:02.362845 20995 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:02.370335 20995 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d2ca21cf24046aeb6bcacf7017bbdcd: Generated new TSK 0
I20260927 18:44:02.370538 20995 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:44:02.403680 21014 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260927 18:44:02.403776 21014 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:44:02.404053 20987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.381051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33956 (local address 127.15.110.254:33377)
0927 18:44:02.381127 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.381129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.381296 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:44:02.381425 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.381426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.381445 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.381520 (+ 75us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:44:02.381688 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.382211 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.382990 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.383252 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.383337 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.383554 (+ 217us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:44:02.403693 (+ 20139us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:44:02.403978 (+ 285us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260927 18:44:02.404121 20987 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33956: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260927 18:44:02.404433 21013 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:02.380991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33377 (local address 127.0.0.1:33956)
0927 18:44:02.381224 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.381234 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:02.381314 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:02.381529 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:02.381530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:02.381532 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:44:02.381614 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.381616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.382300 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.382301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.382900 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.382902 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.383341 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.383342 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.383410 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.383436 (+ 26us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:44:02.403891 (+ 20455us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0927 18:44:02.404016 (+ 125us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.110.254:33377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":103,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.15.110.254:33377: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:37597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260927 18:44:02.404837 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 20917
2026-09-27T18:44:02Z chronyd exiting
I20260927 18:44:02.441159 15803 test_util.cc:175] -----------------------------------------------
I20260927 18:44:02.441241 15803 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790534620099873-15803-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (247 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260927 18:44:02.441972 15803 mini_oidc.cc:152] Starting JWKS server
I20260927 18:44:02.442656 15803 webserver.cc:460] Webserver started at https://127.0.0.1:42517/ using document root <none> and password file <none>
I20260927 18:44:02.442698 15803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:44:02.442839 15803 webserver.cc:460] Webserver started at http://127.0.0.1:36655/ using document root <none> and password file <none>
2026-09-27T18:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:02Z Disabled control of system clock
I20260927 18:44:02.450371 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33759
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:35681
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:33759
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42517/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/testchainca.cert with env {}
W20260927 18:44:02.530539 21020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:02.530758 21020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:02.530781 21020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:02.531919 21020 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:02.531960 21020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:02.531972 21020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:02.531986 21020 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:02.531998 21020 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:44:02.532011 21020 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:44:02.532022 21020 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42517/jwks.json
I20260927 18:44:02.533502 21020 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35681
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:33759
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42517/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:02.533700 21020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:02.533948 21020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:02.538916 21025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.538904 21026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.538931 21020 server_base.cc:1034] running on GCE node
W20260927 18:44:02.538904 21028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.539417 21020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:02.539719 21020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:02.540884 21020 hybrid_clock.cc:648] HybridClock initialized: now 1790534642540879 us; error 36 us; skew 500 ppm
I20260927 18:44:02.542063 21020 webserver.cc:460] Webserver started at http://127.15.110.254:36659/ using document root <none> and password file <none>
I20260927 18:44:02.542364 21020 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:02.542456 21020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:02.542618 21020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:02.543848 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "3ddf29fe93c14996947ab193c049683a"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "5356e115b9cf02c47e90a24ab0c0e957"
server_key_iv: "5640e0254f179e8a0a569da37a9d4c74"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.544241 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "3ddf29fe93c14996947ab193c049683a"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "5356e115b9cf02c47e90a24ab0c0e957"
server_key_iv: "5640e0254f179e8a0a569da37a9d4c74"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.545614 21020 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:44:02.546355 21034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:02.546533 21020 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:02.546630 21020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "3ddf29fe93c14996947ab193c049683a"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "5356e115b9cf02c47e90a24ab0c0e957"
server_key_iv: "5640e0254f179e8a0a569da37a9d4c74"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.546737 21020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:02.554744 21020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:02.555017 21020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:02.555164 21020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:02.559135 21020 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:33759
I20260927 18:44:02.559166 21086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:33759 every 8 connection(s)
I20260927 18:44:02.559547 21020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:02.560195 21087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:02.562956 21087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a: Bootstrap starting.
I20260927 18:44:02.563592 21087 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:02.563879 21087 log.cc:824] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:02.564373 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21020
I20260927 18:44:02.564463 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:02.564646 15803 external_mini_cluster.cc:1428] Setting key 797ccb3f93e528ee54ba88609aeac37d
I20260927 18:44:02.564646 21087 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a: No bootstrap required, opened a new log
I20260927 18:44:02.567247 21087 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } }
I20260927 18:44:02.567409 21087 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:02.567452 21087 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ddf29fe93c14996947ab193c049683a, State: Initialized, Role: FOLLOWER
I20260927 18:44:02.567579 21087 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } }
I20260927 18:44:02.567680 21087 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:02.567720 21087 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:02.567771 21087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.568312 21087 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } }
I20260927 18:44:02.568430 21087 leader_election.cc:302] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ddf29fe93c14996947ab193c049683a; no voters:
I20260927 18:44:02.568583 21087 leader_election.cc:288] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:02.568614 21092 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:02.568715 21092 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [term 1 LEADER]: Becoming Leader. State: Replica: 3ddf29fe93c14996947ab193c049683a, State: Running, Role: LEADER
I20260927 18:44:02.568801 21092 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } }
I20260927 18:44:02.568843 21087 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:02.569119 21092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ddf29fe93c14996947ab193c049683a. Latest consensus state: current_term: 1 leader_uuid: "3ddf29fe93c14996947ab193c049683a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } } }
I20260927 18:44:02.569128 21093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ddf29fe93c14996947ab193c049683a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ddf29fe93c14996947ab193c049683a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33759 } } }
I20260927 18:44:02.569200 21092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.569334 21093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.569829 21101 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:02.570387 21101 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:02.570631 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.565150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38110 (local address 127.15.110.254:33759)
0927 18:44:02.565346 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.565353 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.565377 (+ 24us) server_negotiation.cc:407] Connection header received
0927 18:44:02.565936 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.565942 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.565998 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.566109 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.566335 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.567357 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.568270 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.568858 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.569293 (+ 435us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.569750 (+ 457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.569754 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.569755 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.569959 (+ 204us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.569998 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.570014 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.570072 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.570340 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.570420 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":81,"threads_started":1}
I20260927 18:44:02.571556 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:44:02.572563 21101 catalog_manager.cc:1347] Generated new cluster ID: 7d4d7e15b6b14173b59503f52bf76a05
I20260927 18:44:02.572618 21101 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:02.588033 21101 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:02.588582 21101 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:02.593070 21101 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ddf29fe93c14996947ab193c049683a: Generated new TSK 0
I20260927 18:44:02.593261 21101 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260927 18:44:02.615100 21117 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260927 18:44:02.615211 21117 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260927 18:44:02.615553 21091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.600884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38114 (local address 127.15.110.254:33759)
0927 18:44:02.600924 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.600926 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.601028 (+ 102us) server_negotiation.cc:407] Connection header received
0927 18:44:02.601081 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.601082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.601095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.601175 (+ 80us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:44:02.601333 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.602136 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.602904 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.603246 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.603316 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.603461 (+ 145us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:44:02.615327 (+ 11866us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:44:02.615482 (+ 155us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 18:44:02.615612 21091 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38114: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260927 18:44:02.615726 21116 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:02.600787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33759 (local address 127.0.0.1:38114)
0927 18:44:02.600966 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.600974 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:02.601026 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:02.601206 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:02.601207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:02.601209 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:44:02.601262 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.601264 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.602231 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.602232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.602828 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.602830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.603323 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.603324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.603365 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.603370 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:44:02.615448 (+ 12078us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0927 18:44:02.615587 (+ 139us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.110.254:33759: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42517/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":92,"threads_started":1}
I20260927 18:44:02.616120 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21020
2026-09-27T18:44:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (211 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260927 18:44:02.653703 15803 mini_oidc.cc:152] Starting JWKS server
I20260927 18:44:02.654397 15803 webserver.cc:460] Webserver started at https://127.0.0.1:44829/ using document root <none> and password file <none>
I20260927 18:44:02.654436 15803 mini_oidc.cc:189] Starting OIDC Discovery server
I20260927 18:44:02.654623 15803 webserver.cc:460] Webserver started at http://127.0.0.1:46701/ using document root <none> and password file <none>
2026-09-27T18:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:02Z Disabled control of system clock
I20260927 18:44:02.662163 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:34763
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:44027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:34763
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44829/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:44:02.742604 21123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:02.742866 21123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:02.742902 21123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:02.744154 21123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:02.744230 21123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:02.744251 21123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:02.744273 21123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:02.744302 21123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:44:02.744325 21123 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:44:02.744349 21123 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44829/jwks.json
I20260927 18:44:02.745708 21123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:34763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:34763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44829/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:02.745985 21123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:02.746285 21123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:02.751497 21129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.751492 21128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.751600 21123 server_base.cc:1034] running on GCE node
W20260927 18:44:02.751492 21131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.751901 21123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:02.752151 21123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:02.753300 21123 hybrid_clock.cc:648] HybridClock initialized: now 1790534642753285 us; error 27 us; skew 500 ppm
I20260927 18:44:02.754688 21123 webserver.cc:460] Webserver started at http://127.15.110.254:36983/ using document root <none> and password file <none>
I20260927 18:44:02.754943 21123 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:02.754990 21123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:02.755136 21123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:02.756273 21123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "49fabecbecb94873aefd5c4f838f215e"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "e25d8f36b9ce8ece506a7399ee672448"
server_key_iv: "e69cc25f685b10c157c1f2b50e316b42"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.756630 21123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "49fabecbecb94873aefd5c4f838f215e"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "e25d8f36b9ce8ece506a7399ee672448"
server_key_iv: "e69cc25f685b10c157c1f2b50e316b42"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.757922 21123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:02.758672 21137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:02.758886 21123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:02.758996 21123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "49fabecbecb94873aefd5c4f838f215e"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "e25d8f36b9ce8ece506a7399ee672448"
server_key_iv: "e69cc25f685b10c157c1f2b50e316b42"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.759075 21123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:02.771780 21123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:02.772120 21123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:02.772296 21123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:02.776372 21123 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:34763
I20260927 18:44:02.776451 21189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:34763 every 8 connection(s)
I20260927 18:44:02.776808 21123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:02.777474 21190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:02.780267 21190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e: Bootstrap starting.
I20260927 18:44:02.780814 21190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:02.781039 21190 log.cc:824] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:02.781677 21190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e: No bootstrap required, opened a new log
I20260927 18:44:02.783308 21190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } }
I20260927 18:44:02.783447 21190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:02.783469 21190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49fabecbecb94873aefd5c4f838f215e, State: Initialized, Role: FOLLOWER
I20260927 18:44:02.783571 21190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } }
I20260927 18:44:02.783622 21190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:02.783643 21190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:02.783669 21190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.784171 21190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } }
I20260927 18:44:02.784255 21190 leader_election.cc:302] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49fabecbecb94873aefd5c4f838f215e; no voters:
I20260927 18:44:02.784380 21190 leader_election.cc:288] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:02.784451 21193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:02.784610 21193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [term 1 LEADER]: Becoming Leader. State: Replica: 49fabecbecb94873aefd5c4f838f215e, State: Running, Role: LEADER
I20260927 18:44:02.784638 21190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:02.784751 21193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } }
I20260927 18:44:02.785095 21194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49fabecbecb94873aefd5c4f838f215e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } } }
I20260927 18:44:02.785250 21194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.785274 21195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49fabecbecb94873aefd5c4f838f215e. Latest consensus state: current_term: 1 leader_uuid: "49fabecbecb94873aefd5c4f838f215e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49fabecbecb94873aefd5c4f838f215e" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 34763 } } }
I20260927 18:44:02.785383 21195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.785771 21202 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:02.786252 21202 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:02.786434 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21123
I20260927 18:44:02.786530 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:02.786693 15803 external_mini_cluster.cc:1428] Setting key c877a51c93e4a4e47a4059b3c44d0e62
I20260927 18:44:02.787912 21202 catalog_manager.cc:1347] Generated new cluster ID: 04ddd1367a244613a136c1020596619a
I20260927 18:44:02.787966 21202 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:02.792891 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.787100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52434 (local address 127.15.110.254:34763)
0927 18:44:02.787369 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.787377 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.787431 (+ 54us) server_negotiation.cc:407] Connection header received
0927 18:44:02.787596 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.787603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.787676 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.788072 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.788176 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.789563 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.790651 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.791338 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.791471 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.792164 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.792168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.792169 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.792384 (+ 215us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.792440 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.792455 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.792525 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.792596 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.792676 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":105,"threads_started":1}
I20260927 18:44:02.794181 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:44:02.802811 21202 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:02.803555 21202 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:02.808544 21202 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49fabecbecb94873aefd5c4f838f215e: Generated new TSK 0
I20260927 18:44:02.808768 21202 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:02.821290 21212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.817480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52438 (local address 127.15.110.254:34763)
0927 18:44:02.817532 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.817534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.817632 (+ 98us) server_negotiation.cc:407] Connection header received
0927 18:44:02.817715 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.817715 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.817727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.817791 (+ 64us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:44:02.817940 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.818460 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.819625 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.819843 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.819898 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.820085 (+ 187us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:44:02.821067 (+ 982us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44829/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:44:02.821206 (+ 139us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52438: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44829/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":27}
W20260927 18:44:02.821398 21219 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:02.817377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:34763 (local address 127.0.0.1:52438)
0927 18:44:02.817557 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:02.817568 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:02.817638 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:02.817809 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:02.817810 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:02.817812 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:44:02.817873 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.817876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.818538 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.818539 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.819534 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:02.819538 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.819921 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.819922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:02.819981 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.819987 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:44:02.821183 (+ 1196us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44829/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0927 18:44:02.821288 (+ 105us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.110.254:34763: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44829/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":109,"threads_started":1}
I20260927 18:44:02.821631 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21123
E20260927 18:44:02.833751 21220 webserver.cc:533] Webserver: sslize failed:
E20260927 18:44:02.833856 21220 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-27T18:44:02Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (206 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-27T18:44:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:02Z Disabled control of system clock
I20260927 18:44:02.867295 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43085
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:36205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:43085
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/testchainca.cert with env {}
W20260927 18:44:02.947707 21224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:02.947911 21224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:02.947935 21224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:02.949167 21224 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:02.949214 21224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:02.949227 21224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:02.949272 21224 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:02.949299 21224 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260927 18:44:02.949321 21224 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260927 18:44:02.949343 21224 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260927 18:44:02.950896 21224 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:43085
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:02.951166 21224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:02.951429 21224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:02.956387 21230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.956442 21229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:02.956461 21232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:02.956446 21224 server_base.cc:1034] running on GCE node
I20260927 18:44:02.956914 21224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:02.957198 21224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:02.958362 21224 hybrid_clock.cc:648] HybridClock initialized: now 1790534642958360 us; error 40 us; skew 500 ppm
I20260927 18:44:02.959718 21224 webserver.cc:460] Webserver started at http://127.15.110.254:38035/ using document root <none> and password file <none>
I20260927 18:44:02.959981 21224 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:02.960050 21224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:02.960198 21224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:02.961272 21224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "5198deeb7f124082930f0aef9100e8ee"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "dbe35c395485fad9401a92019b97cc82"
server_key_iv: "ff46d31e475ca8cbe6f67a5179cfcaf4"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.961658 21224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "5198deeb7f124082930f0aef9100e8ee"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "dbe35c395485fad9401a92019b97cc82"
server_key_iv: "ff46d31e475ca8cbe6f67a5179cfcaf4"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.962937 21224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:02.963714 21238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:02.963881 21224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:02.963985 21224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "5198deeb7f124082930f0aef9100e8ee"
format_stamp: "Formatted at 2026-09-27 18:44:02 on dist-test-slave-vs05"
server_key: "dbe35c395485fad9401a92019b97cc82"
server_key_iv: "ff46d31e475ca8cbe6f67a5179cfcaf4"
server_key_version: "encryptionkey@0"
I20260927 18:44:02.964061 21224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:02.976936 21224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:02.977267 21224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:02.977427 21224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:02.981431 21224 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:43085
I20260927 18:44:02.981491 21290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:43085 every 8 connection(s)
I20260927 18:44:02.981853 21224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:02.982802 21291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:02.985095 21291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee: Bootstrap starting.
I20260927 18:44:02.985617 21291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:02.985848 21291 log.cc:824] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:02.986478 21291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee: No bootstrap required, opened a new log
I20260927 18:44:02.988091 21291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } }
I20260927 18:44:02.988245 21291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:02.988269 21291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5198deeb7f124082930f0aef9100e8ee, State: Initialized, Role: FOLLOWER
I20260927 18:44:02.988413 21291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } }
I20260927 18:44:02.988498 21291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:02.988523 21291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:02.988550 21291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:02.989006 21291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } }
I20260927 18:44:02.989086 21291 leader_election.cc:302] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5198deeb7f124082930f0aef9100e8ee; no voters:
I20260927 18:44:02.989208 21291 leader_election.cc:288] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:02.989285 21294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:02.989445 21294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [term 1 LEADER]: Becoming Leader. State: Replica: 5198deeb7f124082930f0aef9100e8ee, State: Running, Role: LEADER
I20260927 18:44:02.989487 21291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:02.989596 21294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } }
I20260927 18:44:02.990160 21295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5198deeb7f124082930f0aef9100e8ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } } }
I20260927 18:44:02.990273 21295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.990386 21296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5198deeb7f124082930f0aef9100e8ee. Latest consensus state: current_term: 1 leader_uuid: "5198deeb7f124082930f0aef9100e8ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5198deeb7f124082930f0aef9100e8ee" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43085 } } }
I20260927 18:44:02.990470 21296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:02.990515 21303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:02.991057 21303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:02.991779 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21224
I20260927 18:44:02.991875 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:02.992074 15803 external_mini_cluster.cc:1428] Setting key f1c976137eafd0f36a30b82bb1bde6a8
I20260927 18:44:02.993278 21303 catalog_manager.cc:1347] Generated new cluster ID: d2eece8932114631a127a0811db5af97
I20260927 18:44:02.993331 21303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:02.998852 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:02.992673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.15.110.254:43085)
0927 18:44:02.993338 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:02.993346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:02.993377 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:44:02.993501 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:02.993508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:02.993580 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:02.993723 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:02.993964 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.995403 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.996446 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:02.997085 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:02.997213 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:02.997920 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:02.997924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:02.997924 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:02.998130 (+ 206us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:02.998175 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:02.998191 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:02.998264 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:02.998333 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:44:02.998515 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":55,"threads_started":1}
I20260927 18:44:02.999748 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:44:03.002316 21303 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:03.002813 21303 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:03.008343 21303 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5198deeb7f124082930f0aef9100e8ee: Generated new TSK 0
I20260927 18:44:03.008522 21303 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:03.474679 21314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:03.016898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.15.110.254:43085)
0927 18:44:03.016947 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:03.016949 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:03.017082 (+ 133us) server_negotiation.cc:407] Connection header received
0927 18:44:03.017167 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:03.017168 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:03.017179 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:03.017234 (+ 55us) server_negotiation.cc:226] Negotiated authn=JWT
0927 18:44:03.017360 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:03.017855 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:03.018997 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:03.019265 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:03.019320 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:03.019497 (+ 177us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0927 18:44:03.474442 (+454945us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:44:03.474619 (+ 177us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59226: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 18:44:03.474745 21314 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59226: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260927 18:44:03.474876 21320 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:03.016865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:43085 (local address 127.0.0.1:59226)
0927 18:44:03.017023 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:03.017035 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:03.017097 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:03.017242 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:03.017243 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:03.017244 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0927 18:44:03.017301 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:03.017303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:03.017956 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:03.017957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:03.018891 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:03.018895 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:03.019336 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:03.019337 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:03.019400 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:03.019406 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0927 18:44:03.474586 (+455180us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0927 18:44:03.474710 (+ 124us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.15.110.254:43085: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":106,"threads_started":1}
I20260927 18:44:03.475232 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21224
2026-09-27T18:44:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (632 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (87 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:03 dist-test-slave-vs05 krb5kdc[21332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:03 dist-test-slave-vs05 krb5kdc[21332](info): set up 2 sockets
Sep 27 18:44:03 dist-test-slave-vs05 krb5kdc[21332](info): commencing operation
krb5kdc: starting...
W20260927 18:44:04.719379 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:04 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:44:04.846796 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38457
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:38457
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:04.929412 21345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:04.929675 21345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:04.929716 21345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:04.930944 21345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:04.931020 21345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:04.931047 21345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:04.931078 21345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:04.931103 21345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:04.932636 21345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:38457
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38457
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:04.932924 21345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:04.933164 21345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:04.938467 21353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:04.938525 21350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:04.938504 21351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:04.938899 21345 server_base.cc:1034] running on GCE node
I20260927 18:44:04.939152 21345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:04.939512 21345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:04.940666 21345 hybrid_clock.cc:648] HybridClock initialized: now 1790534644940652 us; error 45 us; skew 500 ppm
Sep 27 18:44:04 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534644, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:04.943279 21345 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:44:04.943666 21345 webserver.cc:460] Webserver started at http://127.15.110.254:43311/ using document root <none> and password file <none>
I20260927 18:44:04.943845 21345 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:04.943884 21345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:04.943980 21345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:04.944998 21345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "529d5d71637b419c8a332d8ebd813f42"
format_stamp: "Formatted at 2026-09-27 18:44:04 on dist-test-slave-vs05"
server_key: "22e3fcc31b037a8c0fd1795f8dc78ea3"
server_key_iv: "21776c1bc7386bc62387ffaaa228dc08"
server_key_version: "encryptionkey@0"
I20260927 18:44:04.945304 21345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "529d5d71637b419c8a332d8ebd813f42"
format_stamp: "Formatted at 2026-09-27 18:44:04 on dist-test-slave-vs05"
server_key: "22e3fcc31b037a8c0fd1795f8dc78ea3"
server_key_iv: "21776c1bc7386bc62387ffaaa228dc08"
server_key_version: "encryptionkey@0"
I20260927 18:44:04.946614 21345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:04.947418 21360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:04.947683 21345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:04.947790 21345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "529d5d71637b419c8a332d8ebd813f42"
format_stamp: "Formatted at 2026-09-27 18:44:04 on dist-test-slave-vs05"
server_key: "22e3fcc31b037a8c0fd1795f8dc78ea3"
server_key_iv: "21776c1bc7386bc62387ffaaa228dc08"
server_key_version: "encryptionkey@0"
I20260927 18:44:04.947865 21345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:04.968276 21345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:04.970741 21345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:04.970924 21345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:04.974768 21345 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:38457
I20260927 18:44:04.974812 21412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:38457 every 8 connection(s)
I20260927 18:44:04.975329 21345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:04.976034 21413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:04.979063 21413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42: Bootstrap starting.
I20260927 18:44:04.979741 21413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:04.980057 21413 log.cc:824] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:04.980809 21413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42: No bootstrap required, opened a new log
I20260927 18:44:04.981953 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21345
I20260927 18:44:04.982061 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:04.982250 15803 external_mini_cluster.cc:1428] Setting key 08c9d6e9312950a625fb5375a7eda489
I20260927 18:44:04.983238 21413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } }
I20260927 18:44:04.983402 21413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:04.983428 21413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 529d5d71637b419c8a332d8ebd813f42, State: Initialized, Role: FOLLOWER
I20260927 18:44:04.983589 21413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } }
I20260927 18:44:04.983690 21413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:04.983731 21413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:04.983784 21413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:04.984344 21413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } }
I20260927 18:44:04.984472 21413 leader_election.cc:302] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 529d5d71637b419c8a332d8ebd813f42; no voters:
I20260927 18:44:04.984692 21413 leader_election.cc:288] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:04.984724 21418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:04.984834 21418 raft_consensus.cc:686] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [term 1 LEADER]: Becoming Leader. State: Replica: 529d5d71637b419c8a332d8ebd813f42, State: Running, Role: LEADER
I20260927 18:44:04.984923 21418 consensus_queue.cc:237] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } }
I20260927 18:44:04.984994 21413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:04.985244 21418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 529d5d71637b419c8a332d8ebd813f42. Latest consensus state: current_term: 1 leader_uuid: "529d5d71637b419c8a332d8ebd813f42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } } }
I20260927 18:44:04.985327 21418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:04.985257 21419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "529d5d71637b419c8a332d8ebd813f42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "529d5d71637b419c8a332d8ebd813f42" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38457 } } }
I20260927 18:44:04.985491 21419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:04.985581 21422 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:04.986155 21422 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:04.987957 21422 catalog_manager.cc:1347] Generated new cluster ID: 13df33db16ad4a7e9b578ea4f59ff48f
I20260927 18:44:04.988014 21422 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:44:04 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534644, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:04.995620 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:04.982883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60148 (local address 127.15.110.254:38457)
0927 18:44:04.983086 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:04.983102 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:04.983129 (+ 27us) server_negotiation.cc:407] Connection header received
0927 18:44:04.983291 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:04.983297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:04.983344 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:04.983514 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:04.984388 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:04.985907 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:04.987310 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:04.987968 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:04.988444 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:04.991170 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:04.991209 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:04.991210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:04.991223 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:04.993448 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:04.993866 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:04.993869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:04.993870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:04.994358 (+ 488us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:04.994587 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:04.994588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:04.994589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:04.994935 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:04.995104 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:04.995216 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 18:44:04.995374 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":119,"threads_started":1}
I20260927 18:44:05.005317 21422 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:05.006297 21422 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:05.020221 21422 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 529d5d71637b419c8a332d8ebd813f42: Generated new TSK 0
I20260927 18:44:05.020431 21422 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:44:05.054582 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:05.149220 21439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:05.149451 21439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:05.149488 21439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:05.150894 21439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:05.150939 21439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:05.150970 21439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:05.152747 21439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21439
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:05.152966 21439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:05.153159 21439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:05.158862 21444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:05.158892 21445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:05.159106 21439 server_base.cc:1034] running on GCE node
W20260927 18:44:05.158862 21447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:05.159395 21439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:05.159646 21439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:05.160804 21439 hybrid_clock.cc:648] HybridClock initialized: now 1790534645160788 us; error 28 us; skew 500 ppm
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.163612 21439 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:44:05.164093 21439 webserver.cc:460] Webserver started at http://127.15.110.193:43633/ using document root <none> and password file <none>
I20260927 18:44:05.164340 21439 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:05.164387 21439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:05.164506 21439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:05.165633 21439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "64cc9e21b3a04f3d8ecfa0738a32e9a4"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "25a5e7c775aac03eb296c17ab12a386b"
server_key_iv: "bd031c6dadd64abb1871a4105c787204"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.165990 21439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "64cc9e21b3a04f3d8ecfa0738a32e9a4"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "25a5e7c775aac03eb296c17ab12a386b"
server_key_iv: "bd031c6dadd64abb1871a4105c787204"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.167449 21439 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:05.168277 21454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.168494 21439 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:05.168599 21439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "64cc9e21b3a04f3d8ecfa0738a32e9a4"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "25a5e7c775aac03eb296c17ab12a386b"
server_key_iv: "bd031c6dadd64abb1871a4105c787204"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.168702 21439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:05.188509 21439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:05.190793 21439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:05.190960 21439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:05.191298 21439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:05.191699 21439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:05.191735 21439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.191794 21439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:05.191823 21439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.198091 21439 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:41189
I20260927 18:44:05.198135 21567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:41189 every 8 connection(s)
I20260927 18:44:05.198499 21439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:05.199970 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21439
I20260927 18:44:05.200084 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:05.200273 15803 external_mini_cluster.cc:1428] Setting key 0f8fcded5f80ea1498bceb509b001241
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:44:05.209350 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.199388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:41621 (local address 127.15.110.254:38457)
0927 18:44:05.199463 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.199465 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.199709 (+ 244us) server_negotiation.cc:407] Connection header received
0927 18:44:05.200858 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.200860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.200872 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.200923 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.201628 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.202190 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.203416 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.203660 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.203743 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.206174 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.206176 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.206176 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.206188 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.207868 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.208340 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.208341 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.208342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.208402 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.208660 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.208661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.208662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.208785 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.208864 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.209159 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.209256 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:44:05.209523 21570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.199375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38457 (local address 127.15.110.193:41621)
0927 18:44:05.199573 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:05.199613 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:05.200736 (+ 1123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:05.201011 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:05.201015 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:05.201304 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:05.201510 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.201515 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.202293 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.202295 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.203311 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.203314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.203762 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.203764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.203906 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.204531 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:05.204540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:05.206055 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:05.208005 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.208012 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.208014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.208249 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.208488 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.208488 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.208489 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.208592 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.208855 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:05.208861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:05.209025 (+ 164us) client_negotiation.cc:769] Sending connection context
0927 18:44:05.209138 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:44:05.209291 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260927 18:44:05.210047 21568 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38457
I20260927 18:44:05.210130 21568 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:05.210281 21568 heartbeater.cc:511] Master 127.15.110.254:38457 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
I20260927 18:44:05.210819 21377 ts_manager.cc:194] Registered new tserver with Master: 64cc9e21b3a04f3d8ecfa0738a32e9a4 (127.15.110.193:41189)
I20260927 18:44:05.211532 21377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:41621
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:44:05.226840 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:05.307898 21573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:05.308125 21573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:05.308149 21573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:05.309327 21573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:05.309369 21573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:05.309396 21573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:05.310832 21573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:05.311105 21573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:05.311385 21573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:44:05.317036 21573 server_base.cc:1034] running on GCE node
W20260927 18:44:05.317020 21581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:05.317018 21579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:05.317018 21578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:05.317485 21573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:05.317731 21573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:05.318915 21573 hybrid_clock.cc:648] HybridClock initialized: now 1790534645318899 us; error 47 us; skew 500 ppm
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.321635 21573 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:44:05.322153 21573 webserver.cc:460] Webserver started at http://127.15.110.194:37067/ using document root <none> and password file <none>
I20260927 18:44:05.322399 21573 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:05.322482 21573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:05.322660 21573 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:05.323818 21573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "b8251a93cf144b0b9063cca6da7103d7"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "e45209e62ad33c31add83a6724638d6a"
server_key_iv: "69cefa61aa3630c438b6ff9dfb1ca6f1"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.324187 21573 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "b8251a93cf144b0b9063cca6da7103d7"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "e45209e62ad33c31add83a6724638d6a"
server_key_iv: "69cefa61aa3630c438b6ff9dfb1ca6f1"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.325529 21573 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:05.326221 21588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.326392 21573 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:05.326498 21573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "b8251a93cf144b0b9063cca6da7103d7"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "e45209e62ad33c31add83a6724638d6a"
server_key_iv: "69cefa61aa3630c438b6ff9dfb1ca6f1"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.326572 21573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:05.345002 21573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:05.347705 21573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:05.347854 21573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:05.348096 21573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:05.348413 21573 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:05.348476 21573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.348518 21573 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:05.348551 21573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.356299 21573 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:37967
I20260927 18:44:05.356371 21701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:37967 every 8 connection(s)
I20260927 18:44:05.356632 21573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:05.362010 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21573
I20260927 18:44:05.362121 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:05.362298 15803 external_mini_cluster.cc:1428] Setting key ce7823cc00f9161b87f2104d0e49a740
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:05.366883 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.357308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:46377 (local address 127.15.110.254:38457)
0927 18:44:05.357380 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.357382 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.357608 (+ 226us) server_negotiation.cc:407] Connection header received
0927 18:44:05.358588 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.358589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.358602 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.358655 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.359352 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.359868 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.360896 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.361115 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.361202 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.363634 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.363635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.363636 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.363645 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.365202 (+ 1557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.365801 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.365802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.365803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.365860 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.366163 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.366164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.366164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.366270 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.366322 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.366675 (+ 353us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.366748 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:44:05.367129 21704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.357308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38457 (local address 127.15.110.194:46377)
0927 18:44:05.357489 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:05.357526 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:05.358466 (+ 940us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:05.358729 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:05.358734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:05.359003 (+ 269us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:05.359244 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.359252 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.359968 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.359969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.360790 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.360792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.361211 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.361212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.361345 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.361883 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:05.361891 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:05.363535 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:05.365309 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.365316 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.365319 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.365708 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.365940 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.365941 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.365941 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.366096 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.366337 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:05.366343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:05.366532 (+ 189us) client_negotiation.cc:769] Sending connection context
0927 18:44:05.366654 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 18:44:05.366837 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":90,"threads_started":1}
I20260927 18:44:05.367817 21702 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38457
I20260927 18:44:05.367937 21702 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:05.368120 21702 heartbeater.cc:511] Master 127.15.110.254:38457 requested a full tablet report, sending...
I20260927 18:44:05.368510 21377 ts_manager.cc:194] Registered new tserver with Master: b8251a93cf144b0b9063cca6da7103d7 (127.15.110.194:37967)
I20260927 18:44:05.368916 21377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:46377
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:44:05.389810 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:05.472620 21707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:05.472924 21707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:05.472966 21707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:05.474298 21707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:05.474365 21707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:05.474411 21707 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:05.476008 21707 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38457
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:05.476290 21707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:05.476528 21707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:05.481899 21713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:05.482034 21707 server_base.cc:1034] running on GCE node
W20260927 18:44:05.481899 21715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:05.482144 21712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:05.482510 21707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:05.482781 21707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:05.483968 21707 hybrid_clock.cc:648] HybridClock initialized: now 1790534645483964 us; error 41 us; skew 500 ppm
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.491098 21707 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:44:05.491652 21707 webserver.cc:460] Webserver started at http://127.15.110.195:35119/ using document root <none> and password file <none>
I20260927 18:44:05.491904 21707 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:05.492012 21707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:05.492170 21707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:05.493283 21707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "bbcb918e156e43a79aa4fc4dd8adbcb1"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "ed057727bdeb8cda06e9c7497edd3128"
server_key_iv: "d2f28287fc1e6421d3bd019e63c7def2"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.493660 21707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "bbcb918e156e43a79aa4fc4dd8adbcb1"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "ed057727bdeb8cda06e9c7497edd3128"
server_key_iv: "d2f28287fc1e6421d3bd019e63c7def2"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.495288 21707 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260927 18:44:05.496156 21722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.496371 21707 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:05.496462 21707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "bbcb918e156e43a79aa4fc4dd8adbcb1"
format_stamp: "Formatted at 2026-09-27 18:44:05 on dist-test-slave-vs05"
server_key: "ed057727bdeb8cda06e9c7497edd3128"
server_key_iv: "d2f28287fc1e6421d3bd019e63c7def2"
server_key_version: "encryptionkey@0"
I20260927 18:44:05.496518 21707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:05.520655 21707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:05.522589 21707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:05.522701 21707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:05.522881 21707 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:05.523270 21707 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:05.523334 21707 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.523376 21707 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:05.523406 21707 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:05.529049 21707 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:33777
I20260927 18:44:05.529127 21835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:33777 every 8 connection(s)
I20260927 18:44:05.529518 21707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:05.535112 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21707
I20260927 18:44:05.535213 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:05.535413 15803 external_mini_cluster.cc:1428] Setting key c72f5d0d97c1a6f02cc3ed6354f71b02
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:05.539104 21417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.530270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:38579 (local address 127.15.110.254:38457)
0927 18:44:05.530350 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.530352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.530577 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:44:05.531618 (+ 1041us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.531620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.531632 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.531688 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.532320 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.532828 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.533856 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.534052 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.534126 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.536739 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.536740 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.536741 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.536747 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.537918 (+ 1171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.538291 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.538292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.538292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.538340 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.538560 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.538560 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.538560 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.538663 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.538714 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.538974 (+ 260us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.539039 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:44:05.539300 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.530257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38457 (local address 127.15.110.195:38579)
0927 18:44:05.530452 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:05.530487 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:05.531510 (+ 1023us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:05.531760 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:05.531764 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:05.532031 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:05.532213 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.532217 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.532920 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.532921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.533721 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:05.533724 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.534145 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.534146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:05.534270 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.534801 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:05.534809 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:05.536664 (+ 1855us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:05.537998 (+ 1334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.538003 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.538005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.538224 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.538404 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:05.538404 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:05.538405 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:05.538499 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:05.538721 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:05.538725 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:05.538841 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 18:44:05.538924 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:44:05.539044 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260927 18:44:05.539793 21836 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38457
I20260927 18:44:05.539909 21836 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:05.540097 21836 heartbeater.cc:511] Master 127.15.110.254:38457 requested a full tablet report, sending...
I20260927 18:44:05.540462 21377 ts_manager.cc:194] Registered new tserver with Master: bbcb918e156e43a79aa4fc4dd8adbcb1 (127.15.110.195:33777)
I20260927 18:44:05.540853 21377 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:38579
I20260927 18:44:05.547314 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/test-admin.keytab.
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.564052 15803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260927 18:44:05.565932 21843 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:05.565044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.15.110.193:41189)
0927 18:44:05.565223 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.565227 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.565237 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:05.565264 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.565266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.565280 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.565349 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.565821 (+ 472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58658: BlockingRecv error: recv got EOF from 127.0.0.1:58658 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":96,"threads_started":1}
W20260927 18:44:05.565959 15803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.566980 15803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.567972 15803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:44:05.576330 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.568504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58662 (local address 127.15.110.193:41189)
0927 18:44:05.568573 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.568575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.568677 (+ 102us) server_negotiation.cc:407] Connection header received
0927 18:44:05.568770 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.568770 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.568782 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.568851 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.569250 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.570050 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.570804 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.571142 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.571300 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.573363 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.573371 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.573371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.573379 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.575113 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.575514 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.575519 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.575520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.575577 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.575779 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.575780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.575781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.575917 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.576108 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.576152 (+ 44us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.576227 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 18:44:05.578166 21843 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:05.577650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58670 (local address 127.15.110.193:41189)
0927 18:44:05.577685 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.577687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.577717 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:44:05.577756 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.577757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.577765 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.577810 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.578117 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58670: BlockingRecv error: recv got EOF from 127.0.0.1:58670 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:44:05.578262 15803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.579231 15803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.580078 15803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:44:05.587524 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.580632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.15.110.193:41189)
0927 18:44:05.580680 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.580682 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.580708 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:44:05.580767 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.580768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.580776 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.580823 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.581183 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.581668 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.582347 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.582553 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.582625 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.584907 (+ 2282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.584909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.584909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.584916 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.586379 (+ 1463us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.586688 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.586689 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.586689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.586748 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.586976 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.586977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.586977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.587064 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.587135 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.587323 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.587385 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260927 18:44:05.589116 21843 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:05.588524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58690 (local address 127.15.110.193:41189)
0927 18:44:05.588585 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.588587 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.588595 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:05.588643 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.588644 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.588652 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.588698 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.589049 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58690: BlockingRecv error: recv got EOF from 127.0.0.1:58690 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
W20260927 18:44:05.589139 15803 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790534620099873-15803-0/krb5kdc/krb5cc' of type FILE
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.590096 15803 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:05.590936 15803 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534645, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:44:05.597859 21843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:05.591571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58700 (local address 127.15.110.193:41189)
0927 18:44:05.591653 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:05.591655 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:05.591677 (+ 22us) server_negotiation.cc:407] Connection header received
0927 18:44:05.591692 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:05.591693 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:05.591700 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:05.591739 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:05.592070 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.592506 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.593169 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:05.593337 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:05.593390 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:05.595508 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:05.595510 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:05.595510 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:05.595517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:05.596804 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.597138 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.597139 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.597139 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.597190 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:05.597365 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:05.597365 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:05.597365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:05.597452 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:05.597521 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:05.597648 (+ 127us) server_negotiation.cc:299] Negotiation successful
0927 18:44:05.597742 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260927 18:44:05.598620 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21439
I20260927 18:44:05.603165 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21573
I20260927 18:44:05.607530 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21707
I20260927 18:44:05.612039 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21345
2026-09-27T18:44:05Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (1962 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21856](info): set up 2 sockets
Sep 27 18:44:05 dist-test-slave-vs05 krb5kdc[21856](info): commencing operation
krb5kdc: starting...
W20260927 18:44:06.667465 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:06 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab.
Entry for principal oryx/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab.
I20260927 18:44:06.792316 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:45065
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab
--principal=oryx/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:45065
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:06.872957 21869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:06.873209 21869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:06.873247 21869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:06.874408 21869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:06.874488 21869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:06.874512 21869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:06.874534 21869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:06.874563 21869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:06.876072 21869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:45065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab
--principal=oryx/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:45065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:06.876353 21869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:06.876590 21869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:06.881718 21874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:06.881801 21877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:06.881845 21875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:06.881747 21869 server_base.cc:1034] running on GCE node
I20260927 18:44:06.882267 21869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:06.882562 21869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:06.883716 21869 hybrid_clock.cc:648] HybridClock initialized: now 1790534646883687 us; error 46 us; skew 500 ppm
Sep 27 18:44:06 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:06.886262 21869 init.cc:377] Logged in from keytab as oryx/127.15.110.254@KRBTEST.COM (short username oryx)
I20260927 18:44:06.886768 21869 webserver.cc:460] Webserver started at http://127.15.110.254:40377/ using document root <none> and password file <none>
I20260927 18:44:06.887015 21869 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:06.887089 21869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:06.887281 21869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:06.888327 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "8140d2b0fc314ba78040641c2faa5aa1"
format_stamp: "Formatted at 2026-09-27 18:44:06 on dist-test-slave-vs05"
server_key: "345b220557aed270b4549daf986f539e"
server_key_iv: "e8da77bbb84b222159cf7511ba6d70d0"
server_key_version: "encryptionkey@0"
I20260927 18:44:06.888690 21869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "8140d2b0fc314ba78040641c2faa5aa1"
format_stamp: "Formatted at 2026-09-27 18:44:06 on dist-test-slave-vs05"
server_key: "345b220557aed270b4549daf986f539e"
server_key_iv: "e8da77bbb84b222159cf7511ba6d70d0"
server_key_version: "encryptionkey@0"
I20260927 18:44:06.890010 21869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:06.890812 21884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:06.891017 21869 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:06.891122 21869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "8140d2b0fc314ba78040641c2faa5aa1"
format_stamp: "Formatted at 2026-09-27 18:44:06 on dist-test-slave-vs05"
server_key: "345b220557aed270b4549daf986f539e"
server_key_iv: "e8da77bbb84b222159cf7511ba6d70d0"
server_key_version: "encryptionkey@0"
I20260927 18:44:06.891235 21869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:06.911319 21869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:06.913254 21869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:06.913422 21869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:06.917459 21869 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:45065
I20260927 18:44:06.917490 21936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:45065 every 8 connection(s)
I20260927 18:44:06.917891 21869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:06.918874 21937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:06.921133 21937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Bootstrap starting.
I20260927 18:44:06.921643 21937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:06.921866 21937 log.cc:824] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:06.922516 21937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: No bootstrap required, opened a new log
I20260927 18:44:06.924065 21937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } }
I20260927 18:44:06.924198 21937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:06.924240 21937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8140d2b0fc314ba78040641c2faa5aa1, State: Initialized, Role: FOLLOWER
I20260927 18:44:06.924348 21937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } }
I20260927 18:44:06.924401 21937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:06.924422 21937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:06.924448 21937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:06.925190 21937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } }
I20260927 18:44:06.925273 21937 leader_election.cc:302] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8140d2b0fc314ba78040641c2faa5aa1; no voters:
I20260927 18:44:06.925391 21937 leader_election.cc:288] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:06.925457 21940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:06.925597 21940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 LEADER]: Becoming Leader. State: Replica: 8140d2b0fc314ba78040641c2faa5aa1, State: Running, Role: LEADER
I20260927 18:44:06.925647 21937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:06.925688 21940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } }
I20260927 18:44:06.926077 21942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8140d2b0fc314ba78040641c2faa5aa1. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } }
I20260927 18:44:06.926122 21941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } }
I20260927 18:44:06.926266 21941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:06.926497 21949 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:06.926669 21942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:06.927019 21949 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:06.927349 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21869
I20260927 18:44:06.927457 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:06.927657 15803 external_mini_cluster.cc:1428] Setting key 1e71082f7d84f85a9e7eb785b24579b4
I20260927 18:44:06.928674 21949 catalog_manager.cc:1347] Generated new cluster ID: 702cd1b0bda9437090a603068f323ec8
I20260927 18:44:06.928725 21949 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:44:06 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:06.939025 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:06.928231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41722 (local address 127.15.110.254:45065)
0927 18:44:06.928436 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:06.928446 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:06.928476 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:44:06.928601 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:06.928608 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:06.928662 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:06.928881 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:06.929571 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:06.931034 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:06.932050 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:06.932685 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:06.932806 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:06.935518 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:06.935522 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:06.935522 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:06.935533 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:06.937450 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:06.937848 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:06.937850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:06.937851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:06.937948 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:06.938156 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:06.938156 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:06.938157 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:06.938429 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:06.938564 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:06.938687 (+ 123us) server_negotiation.cc:299] Negotiation successful
0927 18:44:06.938809 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":83,"threads_started":1}
I20260927 18:44:06.969255 21949 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:06.970028 21949 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:06.974475 21949 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Generated new TSK 0
I20260927 18:44:06.974702 21949 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
I20260927 18:44:07.007001 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:07.095861 21963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:07.096056 21963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:07.096079 21963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:07.097314 21963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:07.097361 21963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:07.097388 21963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:07.098803 21963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:07.099062 21963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:07.099311 21963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:44:07.104458 21963 server_base.cc:1034] running on GCE node
W20260927 18:44:07.104425 21968 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.104461 21969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.104442 21971 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.104833 21963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.105139 21963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.106324 21963 hybrid_clock.cc:648] HybridClock initialized: now 1790534647106297 us; error 37 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.109061 21963 init.cc:377] Logged in from keytab as oryx/127.15.110.193@KRBTEST.COM (short username oryx)
I20260927 18:44:07.109510 21963 webserver.cc:460] Webserver started at http://127.15.110.193:39785/ using document root <none> and password file <none>
I20260927 18:44:07.109714 21963 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.109761 21963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.109849 21963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:07.110839 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "e9511d9b369c42379543d5c449b56c29"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "9c28b81022bbfbd6c4f8a5f4585ca856"
server_key_iv: "2c2f0aae1b1e8b2d98b4837ad8a9971d"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.111160 21963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "e9511d9b369c42379543d5c449b56c29"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "9c28b81022bbfbd6c4f8a5f4585ca856"
server_key_iv: "2c2f0aae1b1e8b2d98b4837ad8a9971d"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.112526 21963 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:07.113252 21978 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.113425 21963 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:07.113548 21963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "e9511d9b369c42379543d5c449b56c29"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "9c28b81022bbfbd6c4f8a5f4585ca856"
server_key_iv: "2c2f0aae1b1e8b2d98b4837ad8a9971d"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.113607 21963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.124804 21963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.127313 21963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:07.127523 21963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.127720 21963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:07.127974 21963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:07.128005 21963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.128027 21963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:07.128041 21963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.133874 21963 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39719
I20260927 18:44:07.133958 22091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39719 every 8 connection(s)
I20260927 18:44:07.134264 21963 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:07.135669 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 21963
I20260927 18:44:07.135775 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:07.135984 15803 external_mini_cluster.cc:1428] Setting key b602923a0891d1fceed28fde7276827c
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:44:07.144975 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.135037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:50889 (local address 127.15.110.254:45065)
0927 18:44:07.135101 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.135103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.135304 (+ 201us) server_negotiation.cc:407] Connection header received
0927 18:44:07.136012 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.136013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.136044 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.136119 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.137371 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.137878 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.138997 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.139305 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.139362 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.141897 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.141898 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.141899 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.141908 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.143485 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.143911 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.143912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.143913 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.143988 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.144252 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.144253 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.144253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.144369 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.144442 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.144737 (+ 295us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.144844 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:44:07.145066 22094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.134992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:50889)
0927 18:44:07.135163 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.135225 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.135889 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.136215 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.136219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.136984 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.137267 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.137274 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.137976 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.137977 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.138879 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.138881 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.139386 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.139387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.139534 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.140073 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.140085 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.141772 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.143577 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.143583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.143586 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.143812 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.144063 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.144064 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.144064 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.144177 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.144444 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.144450 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.144595 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.144704 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.144856 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":94,"threads_started":1}
I20260927 18:44:07.145639 22092 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:07.145736 22092 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:07.145934 22092 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:07.146575 21901 ts_manager.cc:194] Registered new tserver with Master: e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.194@KRBTEST.COM" created.
I20260927 18:44:07.147432 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:50889
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
I20260927 18:44:07.162370 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:07.242473 22097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:07.242668 22097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:07.242691 22097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:07.243789 22097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:07.243824 22097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:07.243850 22097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:07.245188 22097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:07.245361 22097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:07.245538 22097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:07.250439 22102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.250432 22103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.250418 22105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.250420 22097 server_base.cc:1034] running on GCE node
I20260927 18:44:07.250932 22097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.251159 22097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.252355 22097 hybrid_clock.cc:648] HybridClock initialized: now 1790534647252334 us; error 35 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.254813 22097 init.cc:377] Logged in from keytab as oryx/127.15.110.194@KRBTEST.COM (short username oryx)
I20260927 18:44:07.255352 22097 webserver.cc:460] Webserver started at http://127.15.110.194:34535/ using document root <none> and password file <none>
I20260927 18:44:07.255599 22097 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.255668 22097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.255824 22097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:07.256865 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "965f43e36c47407aaf240304e905bde7"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "d66ccc9b7d52a6812627c4f2f027f088"
server_key_iv: "b8f422c1f26bd26b6c90f4b7fb9d469c"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.257248 22097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "965f43e36c47407aaf240304e905bde7"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "d66ccc9b7d52a6812627c4f2f027f088"
server_key_iv: "b8f422c1f26bd26b6c90f4b7fb9d469c"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.258560 22097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:07.259346 22112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.259522 22097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:07.259619 22097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "965f43e36c47407aaf240304e905bde7"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "d66ccc9b7d52a6812627c4f2f027f088"
server_key_iv: "b8f422c1f26bd26b6c90f4b7fb9d469c"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.259706 22097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.272411 22097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.274624 22097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:07.274771 22097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.274998 22097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:07.275326 22097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:07.275380 22097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.275414 22097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:07.275442 22097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.281322 22097 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:45839
I20260927 18:44:07.281394 22225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:45839 every 8 connection(s)
I20260927 18:44:07.281729 22097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:07.287061 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22097
I20260927 18:44:07.287240 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:07.287421 15803 external_mini_cluster.cc:1428] Setting key fc46e6b157788cab0c0deed8da0ddaa2
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:07.291414 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.282527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:57991 (local address 127.15.110.254:45065)
0927 18:44:07.282568 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.282570 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.282759 (+ 189us) server_negotiation.cc:407] Connection header received
0927 18:44:07.283348 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.283349 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.283358 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.283418 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.284363 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.284863 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.285847 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.286027 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.286077 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.288408 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.288410 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.288411 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.288419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.289947 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.290447 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.290448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.290448 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.290497 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.290790 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.290790 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.290790 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.290887 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.290936 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.291283 (+ 347us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.291348 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:44:07.291724 22228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.282461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:57991)
0927 18:44:07.282624 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.282666 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.283158 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.283529 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.283532 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.284093 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.284283 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.284288 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.284963 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.284964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.285766 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.285768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.286104 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.286105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.286208 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.286676 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.286685 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.288316 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.290033 (+ 1717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.290039 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.290042 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.290368 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.290572 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.290574 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.290574 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.290722 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.290954 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.290960 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.291130 (+ 170us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.291268 (+ 138us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.291444 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260927 18:44:07.292294 22226 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:07.292404 22226 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:07.292613 22226 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:07.292994 21901 ts_manager.cc:194] Registered new tserver with Master: 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
I20260927 18:44:07.293426 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:57991
WARNING: no policy specified for oryx/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
I20260927 18:44:07.314322 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:07.403440 22231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:07.403672 22231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:07.403714 22231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:07.405015 22231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:07.405093 22231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:07.405143 22231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:07.406710 22231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:45065
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:07.406996 22231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:07.407301 22231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:07.412688 22239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.412688 22236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.412734 22237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.412931 22231 server_base.cc:1034] running on GCE node
I20260927 18:44:07.413138 22231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.413393 22231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.414533 22231 hybrid_clock.cc:648] HybridClock initialized: now 1790534647414521 us; error 29 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.417078 22231 init.cc:377] Logged in from keytab as oryx/127.15.110.195@KRBTEST.COM (short username oryx)
I20260927 18:44:07.417505 22231 webserver.cc:460] Webserver started at http://127.15.110.195:43597/ using document root <none> and password file <none>
I20260927 18:44:07.417761 22231 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.417833 22231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.418010 22231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:07.419059 22231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "97383684bafb4aae8707a2f2c853affd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "64c092ab62eaefd15db472d1cbb98363"
server_key_iv: "584a6beb605f0414f62f0e689c4f1a24"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.419478 22231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "97383684bafb4aae8707a2f2c853affd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "64c092ab62eaefd15db472d1cbb98363"
server_key_iv: "584a6beb605f0414f62f0e689c4f1a24"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.420790 22231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:07.421679 22246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.421880 22231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:07.421988 22231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "97383684bafb4aae8707a2f2c853affd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "64c092ab62eaefd15db472d1cbb98363"
server_key_iv: "584a6beb605f0414f62f0e689c4f1a24"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.422065 22231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.430997 22231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.433013 22231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:07.433157 22231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.433373 22231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:07.433671 22231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:07.433725 22231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.433768 22231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:07.433795 22231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.439524 22231 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:39085
I20260927 18:44:07.439597 22359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:39085 every 8 connection(s)
I20260927 18:44:07.439924 22231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:07.449206 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.440718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:57969 (local address 127.15.110.254:45065)
0927 18:44:07.440762 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.440764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.440989 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:44:07.441499 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.441500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.441511 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.441562 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.442511 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.443004 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.444099 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.444284 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.444341 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.446558 (+ 2217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.446559 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.446560 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.446567 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.447991 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.448364 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.448365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.448365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.448412 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.448634 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.448634 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.448634 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.448732 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.448783 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.449037 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.449093 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:07.449321 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22231
I20260927 18:44:07.449352 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.440714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:57969)
0927 18:44:07.440869 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.440905 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.441411 (+ 506us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.441617 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.441621 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.442216 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.442412 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.442417 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.443082 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.443083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.444021 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.444023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.444361 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.444362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.444464 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.445040 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.445048 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.446461 (+ 1413us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.448066 (+ 1605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.448072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.448073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.448299 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.448473 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.448474 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.448474 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.448569 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.448794 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.448799 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.448915 (+ 116us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.449011 (+ 96us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.449132 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260927 18:44:07.449393 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:07.449643 15803 external_mini_cluster.cc:1428] Setting key 4eeab88148c0c5fb779e58fbe193a949
I20260927 18:44:07.449867 22360 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:07.450016 22360 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:07.450225 22360 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:07.450691 21901 ts_manager.cc:194] Registered new tserver with Master: 97383684bafb4aae8707a2f2c853affd (127.15.110.195:39085)
I20260927 18:44:07.451025 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:57969
I20260927 18:44:07.461520 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.110.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.253.keytab.
Entry for principal oryx/127.15.110.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.253.keytab.
I20260927 18:44:07.486333 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:40299
--webserver_interface=127.15.110.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.253.keytab
--principal=oryx/127.15.110.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:45065,127.15.110.253:40299
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:07.566293 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:07.566533 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:07.566572 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:07.567750 22365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:07.567816 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:07.567839 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:07.567862 22365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:07.567890 22365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:07.569321 22365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:45065,127.15.110.253:40299
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.253.keytab
--principal=oryx/127.15.110.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:40299
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.110.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
W20260927 18:44:07.569463 22365 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.15.110.254:45065,127.15.110.253:40299'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260927 18:44:07.569630 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:07.569859 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:07.574592 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.574643 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.574594 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.574839 22365 server_base.cc:1034] running on GCE node
I20260927 18:44:07.575104 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.575415 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.576567 22365 hybrid_clock.cc:648] HybridClock initialized: now 1790534647576549 us; error 33 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.579041 22365 init.cc:377] Logged in from keytab as oryx/127.15.110.253@KRBTEST.COM (short username oryx)
I20260927 18:44:07.579514 22365 webserver.cc:460] Webserver started at http://127.15.110.253:40187/ using document root <none> and password file <none>
I20260927 18:44:07.579733 22365 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.579778 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.579933 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:07.581071 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/instance:
uuid: "be114d271da945afa411718cd0a63cdd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "33b437a2335bc9819d8a6aa6909afad7"
server_key_iv: "2d67472f909f5c971655d4625510ed73"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.581437 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal/instance:
uuid: "be114d271da945afa411718cd0a63cdd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "33b437a2335bc9819d8a6aa6909afad7"
server_key_iv: "2d67472f909f5c971655d4625510ed73"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.582783 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:07.583571 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.583726 22365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.583813 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "be114d271da945afa411718cd0a63cdd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "33b437a2335bc9819d8a6aa6909afad7"
server_key_iv: "2d67472f909f5c971655d4625510ed73"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.583868 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.601788 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.603788 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:07.603915 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.607836 22365 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.253:40299
I20260927 18:44:07.607892 22432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.253:40299 every 8 connection(s)
I20260927 18:44:07.608223 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
I20260927 18:44:07.609222 22433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:07.611023 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22365
I20260927 18:44:07.610872 22433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.253@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
I20260927 18:44:07.620785 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.611734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41734 (local address 127.15.110.254:45065)
0927 18:44:07.611785 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.611788 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.611974 (+ 186us) server_negotiation.cc:407] Connection header received
0927 18:44:07.612502 (+ 528us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.612503 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.612517 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.612570 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.613132 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.613693 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.614659 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.614889 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.614979 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.617434 (+ 2455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.617435 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.617436 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.617442 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.619272 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.619733 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.619734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.619735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.619786 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.620051 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.620052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.620052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.620250 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.620325 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.620639 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.620700 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:44:07.621030 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.611680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41734)
0927 18:44:07.611832 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.611880 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.612390 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.612601 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.612604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.612868 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.613032 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.613036 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.613803 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.613804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.614548 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.614550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.614991 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.614992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.615097 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.615805 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.615813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.617351 (+ 1538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.619349 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.619354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.619356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.619657 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.619854 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.619855 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.619856 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.619985 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.620317 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.620320 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.620477 (+ 157us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.620599 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.620775 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":71,"threads_started":1}
I20260927 18:44:07.621461 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.611622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56508 (local address 127.15.110.253:40299)
0927 18:44:07.611805 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.611814 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.611963 (+ 149us) server_negotiation.cc:407] Connection header received
0927 18:44:07.612404 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.612414 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.612434 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.612532 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.613139 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.613944 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.614689 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.614988 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.615100 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.618033 (+ 2933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.618040 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.618040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.618047 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.619900 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.620320 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.620323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.620323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.620379 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.620579 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.620580 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.620581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.620726 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.620867 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.621279 (+ 412us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.621357 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":99,"threads_started":1}
I20260927 18:44:07.621721 22433 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.253@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
I20260927 18:44:07.627943 22437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.621994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56510)
0927 18:44:07.622034 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.622043 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.622075 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.622213 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.622213 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.622338 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.622368 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.622369 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.622958 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.622959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.623517 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.623520 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.623840 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.623841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.623883 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.624282 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.624287 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.625443 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.627029 (+ 1586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.627030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.627030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.627269 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.627434 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.627435 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.627435 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.627489 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.627690 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.627691 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.627726 (+ 35us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.627794 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.627865 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260927 18:44:07.627993 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.622048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56510 (local address 127.15.110.253:40299)
0927 18:44:07.622082 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.622084 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.622093 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:07.622133 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.622134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.622143 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.622190 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.622441 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.622875 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.623587 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.623775 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.623827 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.625518 (+ 1691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.625519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.625519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.625524 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.626948 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.627332 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.627333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.627333 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.627378 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.627536 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.627536 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.627536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.627617 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.627661 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.627810 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.627906 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":20}
I20260927 18:44:07.629415 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Bootstrap starting.
I20260927 18:44:07.630208 22433 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:07.630481 22433 log.cc:824] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:07.631165 22433 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: No bootstrap required, opened a new log
I20260927 18:44:07.632568 22433 raft_consensus.cc:348] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } }
I20260927 18:44:07.632709 22433 raft_consensus.cc:374] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:07.632758 22433 raft_consensus.cc:729] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be114d271da945afa411718cd0a63cdd, State: Initialized, Role: FOLLOWER
I20260927 18:44:07.632889 22433 consensus_queue.cc:260] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } }
I20260927 18:44:07.633199 22433 sys_catalog.cc:564] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:07.633170 22439 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } } }
I20260927 18:44:07.633299 22439 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:44:07.634781 22450 catalog_manager.cc:1558] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:44:07.634835 22450 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:44:07.636673 22365 master_runner.cc:428] Detected that this master be114d271da945afa411718cd0a63cdd is joining an existing cluster
I20260927 18:44:07.636715 22365 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.110.253:40299
I20260927 18:44:07.644902 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.637929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.15.110.253:40299)
0927 18:44:07.637980 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.637981 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.637989 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:07.638027 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.638028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.638038 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.638091 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.638661 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.639163 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640305 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.640611 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.641654 (+ 967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.641655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.641656 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.641663 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.643393 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.643771 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.643772 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.643773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.643832 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.644056 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.644057 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.644058 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.644205 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.644279 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.644737 (+ 458us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.644808 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:44:07.645215 22456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.637722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56514)
0927 18:44:07.637896 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.637907 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.637958 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.638132 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.638133 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.638459 (+ 326us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.638566 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.638569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.639283 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.639284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.640205 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.640207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.640703 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.640765 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.641230 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.641235 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.641548 (+ 313us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.643477 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.643478 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.643479 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.643688 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.643915 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.643916 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.643917 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.643981 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.644578 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.644579 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.644622 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.645018 (+ 396us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.645105 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
I20260927 18:44:07.645751 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.637778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.15.110.254:45065)
0927 18:44:07.637826 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.637829 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.638121 (+ 292us) server_negotiation.cc:407] Connection header received
0927 18:44:07.638160 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.638161 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.638176 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.638244 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.638678 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.639465 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640516 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.640811 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640888 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.642475 (+ 1587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.642477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.642477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.642485 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.644198 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.644568 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.644568 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.644569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.644619 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.644851 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.644852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.644852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.644993 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.645067 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.645564 (+ 497us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.645650 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:44:07.646101 22457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.637732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41746)
0927 18:44:07.638053 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.638062 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.638112 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.638261 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.638262 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.638536 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.638597 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.638598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.639571 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.639572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.640429 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.640430 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.640903 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.640904 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.641294 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.641698 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.641703 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.642072 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.644279 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.644280 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.644280 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.644482 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.644707 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.644707 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.644708 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.644775 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.645101 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.645102 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.645155 (+ 53us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.645862 (+ 707us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.645980 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":89,"threads_started":1}
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.254@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
I20260927 18:44:07.655480 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.648392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56524)
0927 18:44:07.648524 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.648567 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.648641 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.648788 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.648791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.649047 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.649094 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.649096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.649999 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.650000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.650576 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.650578 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.651027 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.651028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.651081 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.651563 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.651567 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.652863 (+ 1296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.654455 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.654461 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.654463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.654654 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.654849 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.654849 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.654850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.654902 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.655149 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.655155 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.655241 (+ 86us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.655319 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.655403 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":70,"threads_started":1}
I20260927 18:44:07.655531 22436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.648434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56524 (local address 127.15.110.253:40299)
0927 18:44:07.648471 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.648473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.648639 (+ 166us) server_negotiation.cc:407] Connection header received
0927 18:44:07.648714 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.648715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.648725 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.648784 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.649169 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.649905 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.650664 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.650952 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.651022 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.652943 (+ 1921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.652945 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.652945 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.652952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.654378 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.654726 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.654727 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.654727 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.654783 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.654963 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.654964 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.654964 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.655087 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.655154 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.655362 (+ 208us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.655433 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:44:07.656113 21901 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.110.253:40299: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "be114d271da945afa411718cd0a63cdd"
member_type: NON_VOTER
last_known_addr {
host: "127.15.110.253"
port: 40299
}
attrs {
promote: true
}
}
dest_uuid: "8140d2b0fc314ba78040641c2faa5aa1"
cas_config_opid_index: -1
I20260927 18:44:07.656425 21901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } }
I20260927 18:44:07.656934 22460 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER be114d271da945afa411718cd0a63cdd (127.15.110.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } }
I20260927 18:44:07.657174 22407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:07.657913 22407 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260927 18:44:07.657950 22460 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.110.253:40299
I20260927 18:44:07.658176 22365 master.cc:537] Master@127.15.110.253:40299 shutting down...
I20260927 18:44:07.658249 22459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } }
I20260927 18:44:07.658327 22459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:07.658413 22460 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:44:07.659227 21887 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260927 18:44:07.659651 21887 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 -> Peer be114d271da945afa411718cd0a63cdd (127.15.110.253:40299): Couldn't send request to peer be114d271da945afa411718cd0a63cdd. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:44:07.661211 22365 raft_consensus.cc:2229] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:07.661310 22365 raft_consensus.cc:2258] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:07.661363 22365 tablet_replica.cc:330] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: stopping tablet replica
I20260927 18:44:07.673760 22365 master.cc:552] Master@127.15.110.253:40299 shutdown complete.
I20260927 18:44:07.673828 22365 master_runner.cc:305] Clearing existing system tablet
I20260927 18:44:07.673954 22365 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.674013 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.674644 22365 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:07.675158 22464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.675405 22365 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.675470 22365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:07.675505 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "be114d271da945afa411718cd0a63cdd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "33b437a2335bc9819d8a6aa6909afad7"
server_key_iv: "2d67472f909f5c971655d4625510ed73"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.676111 22365 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:07.677778 22365 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:44:07.677843 22365 log.cc:1196] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260927 18:44:07.678035 22365 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Deleting consensus metadata
I20260927 18:44:07.678175 22365 master_runner.cc:315] Copying system tablet from 127.15.110.254:45065
I20260927 18:44:07.678817 22365 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: tablet copy: Beginning tablet copy session from remote peer at address 127.15.110.254:45065
I20260927 18:44:07.684805 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.679173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.15.110.254:45065)
0927 18:44:07.679236 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.679238 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.679297 (+ 59us) server_negotiation.cc:407] Connection header received
0927 18:44:07.679377 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.679378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.679389 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.679445 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.679751 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.680257 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.680998 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.681203 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.681288 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.682258 (+ 970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.682259 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.682259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.682266 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.683662 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.684018 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.684020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.684020 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.684078 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.684284 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.684285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.684285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.684399 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.684477 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.684647 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.684717 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:44:07.684818 22470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.679102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41752)
0927 18:44:07.679232 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.679245 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.679313 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.679452 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.679453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.679625 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.679670 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.679672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.680356 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.680357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.680910 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.680912 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.681291 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.681292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.681355 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.681846 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.681851 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.682150 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.683758 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.683759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.683760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.683932 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.684156 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.684157 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.684157 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.684221 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.684468 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.684469 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.684514 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.684602 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.684705 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":97,"threads_started":1}
I20260927 18:44:07.685341 21921 tablet_copy_service.cc:140] P 8140d2b0fc314ba78040641c2faa5aa1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer be114d271da945afa411718cd0a63cdd ({username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:41752)
I20260927 18:44:07.685451 21921 tablet_copy_service.cc:161] P 8140d2b0fc314ba78040641c2faa5aa1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer be114d271da945afa411718cd0a63cdd at {username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:41752: session id = be114d271da945afa411718cd0a63cdd-00000000000000000000000000000000
I20260927 18:44:07.686260 21921 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:44:07.686836 22365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:44:07.687788 22365 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: tablet copy: Starting download of 0 data blocks...
I20260927 18:44:07.687942 22365 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: tablet copy: Starting download of 1 WAL segments...
I20260927 18:44:07.688925 22365 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:44:07.689684 21921 tablet_copy_service.cc:342] P 8140d2b0fc314ba78040641c2faa5aa1: Request end of tablet copy session be114d271da945afa411718cd0a63cdd-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:41752
I20260927 18:44:07.689749 21921 tablet_copy_service.cc:433] P 8140d2b0fc314ba78040641c2faa5aa1: ending tablet copy session be114d271da945afa411718cd0a63cdd-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer be114d271da945afa411718cd0a63cdd
I20260927 18:44:07.692130 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:07.693274 22471 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.693274 22472 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.696255 22365 server_base.cc:1034] running on GCE node
W20260927 18:44:07.696323 22474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.696476 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.696642 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.697763 22365 hybrid_clock.cc:648] HybridClock initialized: now 1790534647697746 us; error 34 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.699002 22365 init.cc:377] Logged in from keytab as oryx/127.15.110.253@KRBTEST.COM (short username oryx)
I20260927 18:44:07.699323 22365 webserver.cc:460] Webserver started at http://127.15.110.253:45993/ using document root <none> and password file <none>
I20260927 18:44:07.699458 22365 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.699494 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.700086 22365 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:07.700590 22481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.700732 22365 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:07.700779 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "be114d271da945afa411718cd0a63cdd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "33b437a2335bc9819d8a6aa6909afad7"
server_key_iv: "2d67472f909f5c971655d4625510ed73"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.700829 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.748346 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.749912 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.753196 22365 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.253:40299
I20260927 18:44:07.753322 22533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.253:40299 every 8 connection(s)
I20260927 18:44:07.753526 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
I20260927 18:44:07.754248 22534 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:44:07.754700 22534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Bootstrap starting.
I20260927 18:44:07.756865 22534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:44:07.757097 22534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Bootstrap complete.
I20260927 18:44:07.757301 22534 raft_consensus.cc:348] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } }
I20260927 18:44:07.757370 22534 raft_consensus.cc:729] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: be114d271da945afa411718cd0a63cdd, State: Initialized, Role: LEARNER
I20260927 18:44:07.757395 22534 consensus_queue.cc:260] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } }
I20260927 18:44:07.757566 22534 sys_catalog.cc:564] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:07.757587 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } }
I20260927 18:44:07.757688 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: LEARNER
I20260927 18:44:07.758960 22548 catalog_manager.cc:1259] Loaded cluster ID: 702cd1b0bda9437090a603068f323ec8
I20260927 18:44:07.759032 22548 catalog_manager.cc:1552] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: loading cluster ID for follower catalog manager: success
I20260927 18:44:07.760084 22548 catalog_manager.cc:1574] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: acquiring CA information for follower catalog manager: success
I20260927 18:44:07.760365 22548 catalog_manager.cc:1602] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:44:07.779814 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.773737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.15.110.253:40299)
0927 18:44:07.773865 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.773867 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.773875 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:07.773899 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.773899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.773908 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.773975 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.774664 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.775332 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.776060 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.776258 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.776314 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.777403 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.777404 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.777404 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.777410 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.778734 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.779051 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.779052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.779052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.779110 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.779383 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.779384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.779384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.779486 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.779564 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.779679 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.779744 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":77,"threads_started":1}
WARNING: no policy specified for oryx/127.15.110.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.252@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.252.keytab.
Entry for principal oryx/127.15.110.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.252.keytab.
I20260927 18:44:07.807087 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36267
--webserver_interface=127.15.110.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.252.keytab
--principal=oryx/127.15.110.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:07.888347 22552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:07.888558 22552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:07.888581 22552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:07.889741 22552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:07.889786 22552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:07.889801 22552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:07.889813 22552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:07.889863 22552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:07.891434 22552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.252.keytab
--principal=oryx/127.15.110.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.110.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:07.891697 22552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:07.891927 22552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:07.896736 22558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.896773 22557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:07.896773 22560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:07.896812 22552 server_base.cc:1034] running on GCE node
I20260927 18:44:07.897192 22552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:07.897472 22552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:07.898615 22552 hybrid_clock.cc:648] HybridClock initialized: now 1790534647898603 us; error 24 us; skew 500 ppm
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:07.901176 22552 init.cc:377] Logged in from keytab as oryx/127.15.110.252@KRBTEST.COM (short username oryx)
I20260927 18:44:07.901671 22552 webserver.cc:460] Webserver started at http://127.15.110.252:35501/ using document root <none> and password file <none>
I20260927 18:44:07.901969 22552 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:07.902045 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:07.902184 22552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:07.903373 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/instance:
uuid: "38588ae5e41e4d43baa3dfb621a7056f"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "c6ba82735e2b149d8cc1f8c0d0611a2b"
server_key_iv: "4dd1fc9106407a6ad1a7e80e48943df0"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.903740 22552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal/instance:
uuid: "38588ae5e41e4d43baa3dfb621a7056f"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "c6ba82735e2b149d8cc1f8c0d0611a2b"
server_key_iv: "4dd1fc9106407a6ad1a7e80e48943df0"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.905125 22552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:44:07.906031 22567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:07.906237 22552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:07.906345 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "38588ae5e41e4d43baa3dfb621a7056f"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "c6ba82735e2b149d8cc1f8c0d0611a2b"
server_key_iv: "4dd1fc9106407a6ad1a7e80e48943df0"
server_key_version: "encryptionkey@0"
I20260927 18:44:07.906445 22552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:07.922569 22552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:07.924964 22552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:07.925130 22552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:07.929585 22552 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.252:36267
I20260927 18:44:07.929631 22619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.252:36267 every 8 connection(s)
I20260927 18:44:07.930007 22552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
I20260927 18:44:07.930936 22620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:07.931939 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22552
I20260927 18:44:07.932745 22620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.252@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
I20260927 18:44:07.943035 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.933914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.15.110.254:45065)
0927 18:44:07.933987 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.933990 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.934206 (+ 216us) server_negotiation.cc:407] Connection header received
0927 18:44:07.934282 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.934283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.934294 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.934348 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.934770 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.935528 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.936829 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.937123 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.937195 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.939652 (+ 2457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.939653 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.939653 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.939660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.941419 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.941879 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.941881 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.941881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.941947 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.942258 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.942258 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.942259 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.942404 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.942496 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.942845 (+ 349us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.942930 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:44:07.943409 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.933729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41766)
0927 18:44:07.934073 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.934127 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.934207 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.934360 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.934362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.934589 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.934678 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.934682 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.935616 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.935617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.936737 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.936740 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.937227 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.937228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.937292 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.937810 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.937821 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.939507 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.941500 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.941505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.941507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.941801 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.942018 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.942019 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.942019 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.942177 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.942537 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.942544 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.942714 (+ 170us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.942797 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.942956 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":229,"threads_started":1}
I20260927 18:44:07.943415 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.932563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56300 (local address 127.15.110.252:36267)
0927 18:44:07.932733 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.932741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.932767 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:44:07.933251 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.933258 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.933303 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.933533 (+ 230us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.934278 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.935620 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.936665 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.937112 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.937229 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.940159 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.940167 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.940167 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.940174 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.942014 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.942377 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.942380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.942380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.942451 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.942671 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.942672 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.942672 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.942830 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.942967 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.943078 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.943196 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":72,"threads_started":1}
I20260927 18:44:07.944181 22620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.252@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
I20260927 18:44:07.949973 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.944499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56566 (local address 127.15.110.253:40299)
0927 18:44:07.944551 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.944552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.944561 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:07.944593 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.944594 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.944604 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.944654 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.944866 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.945343 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.946015 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.946205 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.946252 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.947866 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.947867 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.947868 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.947874 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.949033 (+ 1159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.949324 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.949325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.949325 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.949370 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.949597 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.949597 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.949597 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.949690 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.949737 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.949854 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.949914 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260927 18:44:07.949975 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.944433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56566)
0927 18:44:07.944480 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.944489 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.944524 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.944664 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.944665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.944770 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.944806 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.944807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.945411 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.945412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.945944 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.945946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.946263 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.946264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.946308 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.946625 (+ 317us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.946629 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.947792 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.949108 (+ 1316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.949109 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.949109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.949260 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.949426 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.949426 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.949426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.949516 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.949748 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.949749 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.949783 (+ 34us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.949836 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.949912 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 18:44:07.950625 22620 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534647, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.252@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
I20260927 18:44:07.956947 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.950896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56302)
0927 18:44:07.950948 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.950965 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.951005 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.951141 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.951142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.951305 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.951331 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.951332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.952244 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.952245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.952708 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.952710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.953033 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.953034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.953085 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.953547 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.953558 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.954652 (+ 1094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.956133 (+ 1481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.956134 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.956134 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.956291 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.956460 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.956460 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.956461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.956509 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.956709 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.956709 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.956742 (+ 33us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.956795 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.956865 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:44:07.956995 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.950942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.15.110.252:36267)
0927 18:44:07.950977 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.950983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.950993 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:07.951065 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.951066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.951074 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.951153 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.951416 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.952173 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.952775 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.952956 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.953003 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.954734 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.954736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.954736 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.954741 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.956054 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.956356 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.956357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.956357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.956404 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.956565 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.956566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.956566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.956656 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.956702 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.956810 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.956906 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":44,"server-negotiator.queue_time_us":17}
I20260927 18:44:07.958663 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Bootstrap starting.
I20260927 18:44:07.959585 22620 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:07.959879 22620 log.cc:824] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:07.960587 22620 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: No bootstrap required, opened a new log
I20260927 18:44:07.961988 22620 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } }
I20260927 18:44:07.962109 22620 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:07.962158 22620 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38588ae5e41e4d43baa3dfb621a7056f, State: Initialized, Role: FOLLOWER
I20260927 18:44:07.962302 22620 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } }
I20260927 18:44:07.962617 22625 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } } }
I20260927 18:44:07.962663 22620 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:07.962729 22625 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:44:07.964557 22636 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:44:07.964627 22636 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:44:07.966908 22552 master_runner.cc:428] Detected that this master 38588ae5e41e4d43baa3dfb621a7056f is joining an existing cluster
I20260927 18:44:07.966957 22552 master_runner.cc:432] Initiating AddMaster RPC to add 127.15.110.252:36267
I20260927 18:44:07.974838 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41772 (local address 127.15.110.254:45065)
0927 18:44:07.968277 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.968279 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.968436 (+ 157us) server_negotiation.cc:407] Connection header received
0927 18:44:07.968590 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.968591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.968604 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.968636 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.969197 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.969691 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.970767 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.971031 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.971063 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.972129 (+ 1066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.972130 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.972131 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.972137 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:07.973613 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.974029 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.974030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.974031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.974096 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:07.974293 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:07.974294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:07.974295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:07.974415 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:07.974508 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:07.974661 (+ 153us) server_negotiation.cc:299] Negotiation successful
0927 18:44:07.974738 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:44:07.974854 22643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41772)
0927 18:44:07.968356 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.968370 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.968504 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.968695 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.968695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.968981 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.969096 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.969098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.969774 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.969776 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.970671 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.970673 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.971133 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.971134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.971250 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.971711 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.971716 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.972026 (+ 310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:07.973730 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.973731 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.973731 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.973921 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.974163 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:07.974164 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:07.974164 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:07.974227 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:07.974500 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:07.974501 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:07.974546 (+ 45us) client_negotiation.cc:769] Sending connection context
0927 18:44:07.974636 (+ 90us) client_negotiation.cc:240] Negotiation successful
0927 18:44:07.974740 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":33,"thread_start_us":78,"threads_started":1}
Sep 27 18:44:07 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534646, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.254@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
I20260927 18:44:07.994942 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21963
I20260927 18:44:08.000315 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:39719
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:45065
--webserver_port=39785
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:44:08.004413 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56580 (local address 127.15.110.253:40299)
0927 18:44:07.968604 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.968606 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.968637 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:44:07.968659 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.968660 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.968669 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.968721 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.969179 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.969922 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.970894 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.971244 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.971315 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.001215 (+ 29900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.001217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.001218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.001226 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.002933 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.003486 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.003487 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.003488 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.003548 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.003755 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.003756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.003757 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.003908 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.003961 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.004168 (+ 207us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.004262 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:08.004791 22644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56580)
0927 18:44:07.968551 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.968560 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.968600 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.968738 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.968739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.968978 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.969096 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.969098 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.970003 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.970004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.970792 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.970793 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.971369 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.971370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.971434 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.000086 (+ 28652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.000093 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.001085 (+ 992us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.003027 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.003028 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.003029 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.003394 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.003617 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.003618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.003618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.003691 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.004009 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.004010 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.004060 (+ 50us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.004520 (+ 460us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.004667 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":6,"thread_start_us":77,"threads_started":1}
I20260927 18:44:08.004973 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.15.110.252:36267)
0927 18:44:07.968499 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.968501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.975310 (+ 6809us) server_negotiation.cc:407] Connection header received
0927 18:44:07.975339 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.975340 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.975351 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.975407 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.975780 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.976447 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.977570 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.977844 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.977909 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.001094 (+ 23185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.001096 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.001096 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.001102 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.003725 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.004064 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.004065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.004065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.004118 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.004288 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.004288 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.004289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.004398 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.004434 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.004812 (+ 378us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.004883 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:44:08.006122 22642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.968420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56318)
0927 18:44:07.975225 (+ 6805us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.975236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.975298 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.975426 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.975427 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.975630 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.975682 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.975684 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.976528 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.976529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.977507 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.977509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.977938 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.977939 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.978003 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.000251 (+ 22248us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.000258 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.000985 (+ 727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.003809 (+ 2824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.003810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.003810 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.003995 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.004180 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.004181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.004181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.004229 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.004470 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.004470 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.004704 (+ 234us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.005898 (+ 1194us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.006007 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6746,"thread_start_us":59,"threads_started":1}
I20260927 18:44:08.006546 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.976892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56320)
0927 18:44:07.976934 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:07.976946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:07.976991 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:07.977264 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:07.977265 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:07.977413 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:07.977448 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.977450 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.978227 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.978228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.978729 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:07.978731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.979216 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.979217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:07.979285 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.979652 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:07.979656 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:07.980824 (+ 1168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.005309 (+ 24485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.005311 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.005312 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.005570 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.005817 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.005818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.005819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.005886 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.006176 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.006177 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.006236 (+ 59us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.006288 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.006403 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260927 18:44:08.006948 22645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:07.976931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56320 (local address 127.15.110.252:36267)
0927 18:44:07.977125 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:07.977127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:07.977134 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:44:07.977172 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:07.977173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:07.977181 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:07.977259 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:07.977526 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.978149 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.978824 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:07.979103 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:07.979182 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:07.980912 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:07.980914 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:07.980914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:07.980921 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.005174 (+ 24253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.005673 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.005673 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.005674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.005733 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.005955 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.005955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.005956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.006094 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.006687 (+ 593us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.006730 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.006810 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":145,"threads_started":1}
I20260927 18:44:08.007411 21901 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.15.110.252:36267: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f"
member_type: NON_VOTER
last_known_addr {
host: "127.15.110.252"
port: 36267
}
attrs {
promote: true
}
}
dest_uuid: "8140d2b0fc314ba78040641c2faa5aa1"
cas_config_opid_index: 5
I20260927 18:44:08.007716 21901 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } }
I20260927 18:44:08.008124 22460 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.009097 22460 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.15.110.252:36267
I20260927 18:44:08.009182 22460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.009241 22460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.009349 22552 master.cc:537] Master@127.15.110.252:36267 shutting down...
I20260927 18:44:08.009485 22594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:08.010531 22594 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 18:44:08.011027 22459 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260927 18:44:08.012562 21887 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 -> Peer 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252:36267): Couldn't send request to peer 38588ae5e41e4d43baa3dfb621a7056f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260927 18:44:08.014124 22552 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:08.014237 22552 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:08.014279 22552 tablet_replica.cc:330] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: stopping tablet replica
I20260927 18:44:08.016383 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.008236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56596)
0927 18:44:08.008419 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.008431 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.008471 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.012629 (+ 4158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.012630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.012632 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.012677 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.012679 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.013694 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.013695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.015071 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.015075 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.016114 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.016115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.016197 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.016223 (+ 26us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.016273 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.016313 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163}
I20260927 18:44:08.016687 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.009318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56596 (local address 127.15.110.253:40299)
0927 18:44:08.012447 (+ 3129us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.012449 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.012458 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:08.012482 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.012482 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.012505 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.012565 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.012761 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.013595 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.015171 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.015939 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.016012 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.016080 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.016575 (+ 495us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.016612 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3104}
I20260927 18:44:08.016894 22508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 LEARNER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260927 18:44:08.017071 22460 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260927 18:44:08.017668 22508 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.018905 22460 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: attempting to promote NON_VOTER be114d271da945afa411718cd0a63cdd to VOTER
I20260927 18:44:08.018963 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8140d2b0fc314ba78040641c2faa5aa1. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.019038 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: LEARNER
I20260927 18:44:08.019070 22537 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: NON_VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: true } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.019106 22537 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: LEARNER
I20260927 18:44:08.019099 22460 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } }
I20260927 18:44:08.019759 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.019629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56328)
0927 18:44:08.019688 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.019733 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260927 18:44:08.019770 22508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 LEARNER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260927 18:44:08.019999 22459 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260927 18:44:08.020272 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.020180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56332)
0927 18:44:08.020224 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.020251 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 18:44:08.020403 21887 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 -> Peer 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252:36267): Couldn't send request to peer 38588ae5e41e4d43baa3dfb621a7056f. Status: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:44:08.020711 22649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be114d271da945afa411718cd0a63cdd (127.15.110.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.020841 22508 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, be114d271da945afa411718cd0a63cdd (127.15.110.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.021788 22648 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.021869 22648 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:44:08.022082 22460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.022162 22460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.022245 22460 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.022293 22460 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.028012 22552 master.cc:552] Master@127.15.110.252:36267 shutdown complete.
I20260927 18:44:08.028069 22552 master_runner.cc:305] Clearing existing system tablet
I20260927 18:44:08.028206 22552 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:08.028254 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:08.028985 22552 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:08.029547 22652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.029719 22552 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:08.029778 22552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:08.029803 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "38588ae5e41e4d43baa3dfb621a7056f"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "c6ba82735e2b149d8cc1f8c0d0611a2b"
server_key_iv: "4dd1fc9106407a6ad1a7e80e48943df0"
server_key_version: "encryptionkey@0"
I20260927 18:44:08.030503 22552 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:08.032178 22552 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260927 18:44:08.032245 22552 log.cc:1196] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260927 18:44:08.032434 22552 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Deleting consensus metadata
I20260927 18:44:08.032613 22552 master_runner.cc:315] Copying system tablet from 127.15.110.254:45065
I20260927 18:44:08.033206 22552 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: tablet copy: Beginning tablet copy session from remote peer at address 127.15.110.254:45065
I20260927 18:44:08.042372 22658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.035446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:41780)
0927 18:44:08.035609 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.035622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.035703 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.035849 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.035850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.036047 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.036101 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.036103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.037042 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.037044 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.037894 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.037897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.038363 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.038364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.038418 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.038935 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.038940 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.039359 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.041248 (+ 1889us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.041249 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.041250 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.041471 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.041697 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.041698 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.041698 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.041762 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.042050 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.042051 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.042098 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.042169 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.042267 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":75,"threads_started":1}
I20260927 18:44:08.042392 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.035505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41780 (local address 127.15.110.254:45065)
0927 18:44:08.035557 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.035559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.035714 (+ 155us) server_negotiation.cc:407] Connection header received
0927 18:44:08.035754 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.035755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.035768 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.035836 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.036188 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.036947 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.037988 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.038277 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.038353 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.039449 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.039451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.039451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.039459 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.041157 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.041555 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.041557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.041557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.041622 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.041835 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.041836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.041836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.041974 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.042052 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.042215 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.042296 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:44:08.043125 21921 tablet_copy_service.cc:140] P 8140d2b0fc314ba78040641c2faa5aa1: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 38588ae5e41e4d43baa3dfb621a7056f ({username='oryx', principal='oryx/127.15.110.252@KRBTEST.COM'} at 127.0.0.1:41780)
I20260927 18:44:08.043234 21921 tablet_copy_service.cc:161] P 8140d2b0fc314ba78040641c2faa5aa1: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 38588ae5e41e4d43baa3dfb621a7056f at {username='oryx', principal='oryx/127.15.110.252@KRBTEST.COM'} at 127.0.0.1:41780: session id = 38588ae5e41e4d43baa3dfb621a7056f-00000000000000000000000000000000
I20260927 18:44:08.043855 21921 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Tablet Copy: opened 0 blocks and 1 log segments
I20260927 18:44:08.044314 22552 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260927 18:44:08.045611 22552 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: tablet copy: Starting download of 0 data blocks...
I20260927 18:44:08.045755 22552 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: tablet copy: Starting download of 1 WAL segments...
I20260927 18:44:08.046484 22552 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260927 18:44:08.047362 21921 tablet_copy_service.cc:342] P 8140d2b0fc314ba78040641c2faa5aa1: Request end of tablet copy session 38588ae5e41e4d43baa3dfb621a7056f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.15.110.252@KRBTEST.COM'} at 127.0.0.1:41780
I20260927 18:44:08.047411 21921 tablet_copy_service.cc:433] P 8140d2b0fc314ba78040641c2faa5aa1: ending tablet copy session 38588ae5e41e4d43baa3dfb621a7056f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 38588ae5e41e4d43baa3dfb621a7056f
I20260927 18:44:08.049893 22552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:08.051049 22659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.051924 22660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.051964 22662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.052450 22552 server_base.cc:1034] running on GCE node
I20260927 18:44:08.052543 22552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:08.052682 22552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:08.053820 22552 hybrid_clock.cc:648] HybridClock initialized: now 1790534648053811 us; error 21 us; skew 500 ppm
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:08.055310 22552 init.cc:377] Logged in from keytab as oryx/127.15.110.252@KRBTEST.COM (short username oryx)
I20260927 18:44:08.055542 22552 webserver.cc:460] Webserver started at http://127.15.110.252:42553/ using document root <none> and password file <none>
I20260927 18:44:08.055662 22552 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:08.055697 22552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:08.056393 22552 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:08.056993 22669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.057176 22552 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.057231 22552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "38588ae5e41e4d43baa3dfb621a7056f"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "c6ba82735e2b149d8cc1f8c0d0611a2b"
server_key_iv: "4dd1fc9106407a6ad1a7e80e48943df0"
server_key_version: "encryptionkey@0"
I20260927 18:44:08.057287 22552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:08.083765 22552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:08.085840 22552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:08.089310 22552 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.252:36267
I20260927 18:44:08.089416 22721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.252:36267 every 8 connection(s)
I20260927 18:44:08.089649 22552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
I20260927 18:44:08.090705 22722 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:44:08.091229 22722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Bootstrap starting.
I20260927 18:44:08.094278 22722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:44:08.094488 22722 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: Bootstrap complete.
I20260927 18:44:08.094700 22722 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } }
I20260927 18:44:08.094789 22722 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 38588ae5e41e4d43baa3dfb621a7056f, State: Initialized, Role: LEARNER
I20260927 18:44:08.094825 22722 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } }
I20260927 18:44:08.095013 22722 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:08.095007 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.095301 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: LEARNER
I20260927 18:44:08.096513 22736 catalog_manager.cc:1259] Loaded cluster ID: 702cd1b0bda9437090a603068f323ec8
I20260927 18:44:08.096598 22736 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: loading cluster ID for follower catalog manager: success
I20260927 18:44:08.097918 22736 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: acquiring CA information for follower catalog manager: success
I20260927 18:44:08.098296 22736 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 18:44:08.107650 22646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:08.107854 22646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:08.107878 22646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:08.109053 22646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:08.109095 22646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:08.109122 22646 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:08.110595 22646 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:39719
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=39785
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:08.110841 22646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:08.111084 22646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:08.116930 22742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.116945 22744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.116930 22741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.117208 22646 server_base.cc:1034] running on GCE node
I20260927 18:44:08.117419 22646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:08.117666 22646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:08.118860 22646 hybrid_clock.cc:648] HybridClock initialized: now 1790534648118843 us; error 30 us; skew 500 ppm
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:08.121635 22646 init.cc:377] Logged in from keytab as oryx/127.15.110.193@KRBTEST.COM (short username oryx)
I20260927 18:44:08.122112 22646 webserver.cc:460] Webserver started at http://127.15.110.193:39785/ using document root <none> and password file <none>
I20260927 18:44:08.122311 22646 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:08.122362 22646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:08.123786 22646 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:44:08.124360 22751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.124558 22646 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:08.124655 22646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "e9511d9b369c42379543d5c449b56c29"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "9c28b81022bbfbd6c4f8a5f4585ca856"
server_key_iv: "2c2f0aae1b1e8b2d98b4837ad8a9971d"
server_key_version: "encryptionkey@0"
I20260927 18:44:08.125005 22646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:08.150260 22646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:08.152761 22646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:08.153273 22646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:08.153469 22646 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:08.153806 22646 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:08.153841 22646 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.153865 22646 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:08.153916 22646 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.160553 22646 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39719
I20260927 18:44:08.160667 22864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39719 every 8 connection(s)
I20260927 18:44:08.161017 22646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:08.166766 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22646
I20260927 18:44:08.166965 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22097
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:08.173456 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.162204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:42055 (local address 127.15.110.253:40299)
0927 18:44:08.162261 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.162264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.163358 (+ 1094us) server_negotiation.cc:407] Connection header received
0927 18:44:08.163458 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.163460 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.163472 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.163502 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.163894 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.164692 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.165789 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.165997 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166076 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.169817 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.169819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.169819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.169828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.171767 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172285 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.172286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.172287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.172347 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172639 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.172640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.172640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.172803 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.172846 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.173220 (+ 374us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.173308 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:44:08.174206 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.162287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:45879 (local address 127.15.110.252:36267)
0927 18:44:08.162420 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.162423 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.162914 (+ 491us) server_negotiation.cc:407] Connection header received
0927 18:44:08.162987 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.162988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.163005 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.163080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.163808 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.165011 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.165836 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.166192 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166320 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.170026 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.170027 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.170028 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.170047 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.172135 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172612 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.172613 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.172614 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.172684 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172925 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.172926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.172926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.173084 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.173919 (+ 835us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.173966 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.174064 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":61,"threads_started":1}
I20260927 18:44:08.174208 22869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.162117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.15.110.193:42055)
0927 18:44:08.163289 (+ 1172us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.163302 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.163386 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.163567 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.163568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.163760 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.163818 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.163820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.164783 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.164783 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.165674 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.165676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.166094 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.166237 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.167007 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.167017 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.169699 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.171859 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.171867 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.171869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.172196 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.172416 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.172417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.172559 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.172890 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.172896 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.173073 (+ 177us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.173553 (+ 480us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.173925 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1153,"mutex_wait_us":237,"thread_start_us":68,"threads_started":1}
I20260927 18:44:08.174489 22871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.161795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.15.110.193:45879)
0927 18:44:08.162415 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.162441 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.162927 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.163102 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.163103 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.163469 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.163694 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.163699 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.165123 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.165124 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.165710 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.165712 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.166327 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166329 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.166401 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.167287 (+ 886us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.167291 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.169926 (+ 2635us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.172244 (+ 2318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172245 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.172246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.172497 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.172769 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.172770 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.172770 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.172845 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.173182 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.173183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.173232 (+ 49us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.173312 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.173840 (+ 528us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":3,"thread_start_us":100,"threads_started":1}
I20260927 18:44:08.174487 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:45839
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:45065
--webserver_port=34535
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:44:08.175419 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.162216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:37697 (local address 127.15.110.254:45065)
0927 18:44:08.162262 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.162264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.162274 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:08.162736 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.162738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.162776 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.162826 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.163839 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.164359 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.165782 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.166110 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166186 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.171743 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.171745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.171746 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.171755 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.173705 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.174269 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.174270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.174271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.174344 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.174724 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.174725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.174726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.174861 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.174913 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.175159 (+ 246us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.175271 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:44:08.175499 22865 heartbeater.cc:347] Connected to a master server at 127.15.110.252:36267
I20260927 18:44:08.175604 22865 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.175724 22870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.161795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:37697)
0927 18:44:08.161976 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.162012 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.162618 (+ 606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.162925 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.162930 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.163673 (+ 743us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.163738 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.163740 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.164459 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.164460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.165675 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.165676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.166186 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.166187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.166256 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.167122 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.167152 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.171622 (+ 4470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.173857 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.173858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.173858 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.174107 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.174590 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.174590 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.174591 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.174650 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.174965 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.174966 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.175010 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.175514 (+ 504us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.175626 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":5,"thread_start_us":100,"threads_started":1}
I20260927 18:44:08.175818 22865 heartbeater.cc:511] Master 127.15.110.252:36267 requested a full tablet report, sending...
I20260927 18:44:08.176196 22867 heartbeater.cc:347] Connected to a master server at 127.15.110.253:40299
I20260927 18:44:08.176254 22867 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.176327 22867 heartbeater.cc:511] Master 127.15.110.253:40299 requested a full tablet report, sending...
I20260927 18:44:08.176471 22686 ts_manager.cc:194] Registered new tserver with Master: e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
I20260927 18:44:08.176692 22866 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:08.176700 22498 ts_manager.cc:194] Registered new tserver with Master: e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
I20260927 18:44:08.176746 22866 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.176806 22866 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:08.177083 21901 ts_manager.cc:194] Re-registered known tserver with Master: e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
I20260927 18:44:08.177469 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:37697
W20260927 18:44:08.259820 22873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:08.260021 22873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:08.260046 22873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:08.261212 22873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:08.261255 22873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:08.261308 22873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:08.262817 22873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:45839
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=34535
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22873
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:08.263079 22873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:08.263362 22873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:08.268291 22878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.268298 22881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.268424 22873 server_base.cc:1034] running on GCE node
W20260927 18:44:08.268298 22879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.268713 22873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:08.268955 22873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:08.270117 22873 hybrid_clock.cc:648] HybridClock initialized: now 1790534648270102 us; error 25 us; skew 500 ppm
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:08.272744 22873 init.cc:377] Logged in from keytab as oryx/127.15.110.194@KRBTEST.COM (short username oryx)
I20260927 18:44:08.273185 22873 webserver.cc:460] Webserver started at http://127.15.110.194:34535/ using document root <none> and password file <none>
I20260927 18:44:08.273377 22873 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:08.273427 22873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:08.274682 22873 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:08.275254 22888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.275439 22873 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:08.275525 22873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "965f43e36c47407aaf240304e905bde7"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "d66ccc9b7d52a6812627c4f2f027f088"
server_key_iv: "b8f422c1f26bd26b6c90f4b7fb9d469c"
server_key_version: "encryptionkey@0"
I20260927 18:44:08.275842 22873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:08.286194 22873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:08.288606 22873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:08.288774 22873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:08.288967 22873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:08.289278 22873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:08.289310 22873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.289332 22873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:08.289377 22873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.295316 22873 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:45839
I20260927 18:44:08.295397 23001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:45839 every 8 connection(s)
I20260927 18:44:08.295773 22873 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:08.300450 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 22873
I20260927 18:44:08.300619 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22231
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
I20260927 18:44:08.308233 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:39085
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:45065
--webserver_port=43597
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:44:08.309691 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.296594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:47735 (local address 127.15.110.254:45065)
0927 18:44:08.296676 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.296680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.296889 (+ 209us) server_negotiation.cc:407] Connection header received
0927 18:44:08.298006 (+ 1117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.298007 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.298021 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.298058 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.298785 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299582 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.300804 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301106 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301144 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.306149 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.306151 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.306152 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.306159 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.307924 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308586 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.308587 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.308587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.308648 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308891 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.308892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.308892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.309024 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.309087 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.309503 (+ 416us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.309584 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:44:08.309691 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.297006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:38429 (local address 127.15.110.253:40299)
0927 18:44:08.297050 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.297052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.297088 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:44:08.298019 (+ 931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.298020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.298033 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.298066 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.298784 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299561 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.300720 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301015 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301093 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.305746 (+ 4653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.305747 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.305748 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.305758 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.307748 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308414 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.308416 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.308417 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.308485 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308811 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.308812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.308812 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.308980 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.309030 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.309490 (+ 460us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.309584 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:44:08.310313 23006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.296577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:47735)
0927 18:44:08.296761 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.296803 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.297902 (+ 1099us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.298145 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.298146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.298530 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.298674 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.298681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299661 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.299661 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.300722 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.300724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301234 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301234 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.301340 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.302075 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.302126 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.306038 (+ 3912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.308014 (+ 1976us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308015 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.308015 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.308490 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.308725 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308725 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.308726 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.308809 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.309135 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.309136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.309281 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.309974 (+ 693us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.310099 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":97,"thread_start_us":98,"threads_started":1}
I20260927 18:44:08.310410 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.296846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:48363 (local address 127.15.110.252:36267)
0927 18:44:08.296890 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.296892 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.296901 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:08.297992 (+ 1091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.297994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.298010 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.298068 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.298779 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299363 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.300747 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301122 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301180 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.305792 (+ 4612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.305793 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.305794 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.305801 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.308658 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.309133 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.309135 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.309135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.309221 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.309451 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.309452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.309453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.309594 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.309641 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.310185 (+ 544us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.310291 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:44:08.310321 23008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.296887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.15.110.194:38429)
0927 18:44:08.297035 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.297049 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.297901 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.298145 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.298146 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.298534 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.298674 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.298681 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299660 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.299661 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.300621 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.300624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301190 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.301340 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.302169 (+ 829us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.302174 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.305634 (+ 3460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.307855 (+ 2221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.307863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.307867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.308306 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.308571 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.308572 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.308725 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.309086 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.309094 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.309281 (+ 187us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.309796 (+ 515us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.310050 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"mutex_wait_us":71,"thread_start_us":77,"threads_started":1}
I20260927 18:44:08.311288 23003 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:08.311388 23003 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.311584 23003 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:08.311859 23004 heartbeater.cc:347] Connected to a master server at 127.15.110.253:40299
I20260927 18:44:08.311910 23004 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.311975 23004 heartbeater.cc:511] Master 127.15.110.253:40299 requested a full tablet report, sending...
I20260927 18:44:08.312008 21901 ts_manager.cc:194] Re-registered known tserver with Master: 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
I20260927 18:44:08.312273 22498 ts_manager.cc:194] Registered new tserver with Master: 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
I20260927 18:44:08.312368 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:47735
I20260927 18:44:08.313011 23007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.296577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.15.110.194:48363)
0927 18:44:08.296817 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.296827 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.297875 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.298133 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.298137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.298470 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.298674 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.298678 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.299463 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.299464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.300628 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.300631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.301286 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.301287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.301378 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.302277 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.302282 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.305690 (+ 3408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.308830 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.308831 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.308832 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.309053 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.309322 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.309322 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.309323 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.309384 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.309691 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.309692 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.309740 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.310523 (+ 783us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.310643 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":5,"thread_start_us":108,"threads_started":1}
I20260927 18:44:08.313377 23002 heartbeater.cc:347] Connected to a master server at 127.15.110.252:36267
I20260927 18:44:08.313444 23002 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.313514 23002 heartbeater.cc:511] Master 127.15.110.252:36267 requested a full tablet report, sending...
I20260927 18:44:08.313934 22686 ts_manager.cc:194] Registered new tserver with Master: 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
W20260927 18:44:08.391995 23009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:08.392186 23009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:08.392210 23009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:08.393360 23009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:08.393440 23009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:08.393488 23009 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:08.395093 23009 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:39085
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=43597
--tserver_master_addrs=127.15.110.254:45065,127.15.110.253:40299,127.15.110.252:36267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:08.395377 23009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:08.395648 23009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:08.400694 23014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:08.400691 23015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.400801 23009 server_base.cc:1034] running on GCE node
W20260927 18:44:08.400969 23017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:08.401175 23009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:08.401386 23009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:08.402522 23009 hybrid_clock.cc:648] HybridClock initialized: now 1790534648402484 us; error 55 us; skew 500 ppm
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:08.404978 23009 init.cc:377] Logged in from keytab as oryx/127.15.110.195@KRBTEST.COM (short username oryx)
I20260927 18:44:08.405448 23009 webserver.cc:460] Webserver started at http://127.15.110.195:43597/ using document root <none> and password file <none>
I20260927 18:44:08.405712 23009 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:08.405797 23009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:08.407169 23009 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:08.407832 23024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.408006 23009 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:08.408107 23009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "97383684bafb4aae8707a2f2c853affd"
format_stamp: "Formatted at 2026-09-27 18:44:07 on dist-test-slave-vs05"
server_key: "64c092ab62eaefd15db472d1cbb98363"
server_key_iv: "584a6beb605f0414f62f0e689c4f1a24"
server_key_version: "encryptionkey@0"
I20260927 18:44:08.408452 23009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:08.418515 23009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:08.420537 23009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:08.420701 23009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:08.420933 23009 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:08.421243 23009 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:08.421298 23009 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.421340 23009 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:08.421368 23009 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:08.427731 23009 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:39085
I20260927 18:44:08.427775 23137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:39085 every 8 connection(s)
I20260927 18:44:08.428146 23009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:08.434361 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23009
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.253@KRBTEST.COM
Sep 27 18:44:08 dist-test-slave-vs05 krb5kdc[21856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534648, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.252@KRBTEST.COM
I20260927 18:44:08.441663 23143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.429318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.15.110.195:42221)
0927 18:44:08.430530 (+ 1212us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.430539 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.430588 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.430709 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.430710 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.431001 (+ 291us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.431270 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.431276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432255 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.432255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433300 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.433302 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433763 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433831 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.435438 (+ 1607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.435443 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.437578 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.439951 (+ 2373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.439960 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.439962 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.440296 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.440517 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.440518 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.440518 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.440654 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.440966 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.440972 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.441143 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.441251 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.441424 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1195,"mutex_wait_us":74,"thread_start_us":59,"threads_started":1}
I20260927 18:44:08.442094 22872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.429391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42221 (local address 127.15.110.252:36267)
0927 18:44:08.429440 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.429442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.430580 (+ 1138us) server_negotiation.cc:407] Connection header received
0927 18:44:08.430649 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.430650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.430660 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.430706 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.431365 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432167 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433387 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433683 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433761 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.438050 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.438051 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.438052 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.438059 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.439850 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.440388 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.440389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.440390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.440446 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.440727 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.440728 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.440729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.440869 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.440941 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.441913 (+ 972us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.441998 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:44:08.442467 23138 heartbeater.cc:347] Connected to a master server at 127.15.110.252:36267
I20260927 18:44:08.442567 23138 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.442767 23138 heartbeater.cc:511] Master 127.15.110.252:36267 requested a full tablet report, sending...
I20260927 18:44:08.442987 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.428860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.15.110.195:35285)
0927 18:44:08.429179 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.429222 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.430341 (+ 1119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.430663 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.430663 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.431023 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.431329 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.431333 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432138 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.432138 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433315 (+ 1177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.433317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433802 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433882 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.434571 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.434575 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.437662 (+ 3087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.440868 (+ 3206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.440869 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.440869 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.441079 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.442170 (+ 1091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.442171 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.442171 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.442231 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.442682 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.442683 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.442724 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.442797 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.442889 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"mutex_wait_us":72,"thread_start_us":242,"threads_started":1}
I20260927 18:44:08.443271 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.428910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:35285 (local address 127.15.110.253:40299)
0927 18:44:08.428954 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.428956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.429328 (+ 372us) server_negotiation.cc:407] Connection header received
0927 18:44:08.430472 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.430473 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.430486 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.430539 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.431446 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432030 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433416 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433686 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433749 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.437741 (+ 3992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.437743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.437743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.437750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.440789 (+ 3039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.442036 (+ 1247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.442037 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.442038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.442092 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.442405 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.442406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.442406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.442565 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.442632 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.443088 (+ 456us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.443196 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:44:08.443298 22686 ts_manager.cc:194] Registered new tserver with Master: 97383684bafb4aae8707a2f2c853affd (127.15.110.195:39085)
I20260927 18:44:08.443758 23140 heartbeater.cc:347] Connected to a master server at 127.15.110.253:40299
I20260927 18:44:08.443806 23140 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.443886 23140 heartbeater.cc:511] Master 127.15.110.253:40299 requested a full tablet report, sending...
I20260927 18:44:08.444180 22498 ts_manager.cc:194] Registered new tserver with Master: 97383684bafb4aae8707a2f2c853affd (127.15.110.195:39085)
I20260927 18:44:08.444829 21960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.428849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:33239 (local address 127.15.110.254:45065)
0927 18:44:08.428924 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.428927 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.429516 (+ 589us) server_negotiation.cc:407] Connection header received
0927 18:44:08.430441 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.430442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.430455 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.430497 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.431482 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432204 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433232 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433521 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433589 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.437852 (+ 4263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.437853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.437854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.437861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.441847 (+ 3986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.444033 (+ 2186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.444034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.444035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.444093 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.444302 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.444303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.444304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.444444 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.444487 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.444658 (+ 171us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.444727 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:44:08.445137 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.428860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:33239)
0927 18:44:08.429430 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.429442 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.430356 (+ 914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.430581 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.430585 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.431018 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:08.431341 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.431343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.432290 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.432291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433131 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.433133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.433604 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.433605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.433736 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.434286 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:08.434295 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:08.437760 (+ 3465us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:08.443688 (+ 5928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.443689 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.443690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.443941 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.444166 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:08.444166 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:08.444166 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:08.444228 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:08.444514 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:08.444515 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:08.444546 (+ 31us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.444620 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.444721 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":533,"mutex_wait_us":253,"thread_start_us":243,"threads_started":1}
I20260927 18:44:08.445438 23139 heartbeater.cc:347] Connected to a master server at 127.15.110.254:45065
I20260927 18:44:08.445483 23139 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:08.445544 23139 heartbeater.cc:511] Master 127.15.110.254:45065 requested a full tablet report, sending...
I20260927 18:44:08.445760 21901 ts_manager.cc:194] Re-registered known tserver with Master: 97383684bafb4aae8707a2f2c853affd (127.15.110.195:39085)
I20260927 18:44:08.446219 21901 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:33239
I20260927 18:44:08.446252 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.435686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56612 (local address 127.15.110.253:40299)
0927 18:44:08.436169 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.436172 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.436182 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:08.436203 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.436204 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.436218 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.436299 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.437145 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.437862 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.438837 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.439131 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.439206 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.441590 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.441591 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.441591 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.441598 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.444092 (+ 2494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.445422 (+ 1330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.445423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.445423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.445480 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.445678 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.445679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.445679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.445799 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.445896 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.446082 (+ 186us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.446163 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":461,"thread_start_us":63,"threads_started":1}
I20260927 18:44:08.446694 23152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.435740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41792 (local address 127.15.110.254:45065)
0927 18:44:08.436229 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.436230 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.436236 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:44:08.436252 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.436253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.436262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.436924 (+ 662us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.436954 (+ 30us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.437721 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.438807 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.439123 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.439523 (+ 400us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.440974 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.440975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.440976 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.440992 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.444277 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.444666 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.444667 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.444667 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.444723 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.445089 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.445090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.445091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.445426 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.446257 (+ 831us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.446512 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.446605 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":74,"threads_started":1}
I20260927 18:44:08.448472 21901 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:08.450021 21901 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:08.450100 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.435950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56344 (local address 127.15.110.252:36267)
0927 18:44:08.439600 (+ 3650us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.439601 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.439607 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:44:08.439625 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.439625 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.439634 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.439718 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.441621 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.443448 (+ 1827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.445132 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.445509 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.445573 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.447450 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.447451 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.447451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.447458 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.448768 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.449131 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.449132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.449133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.449193 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.449392 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.449394 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.449394 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.449520 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.449622 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.449737 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.450015 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3633,"thread_start_us":53,"threads_started":1}
I20260927 18:44:08.457695 22458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.453965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39719 (local address 127.0.0.1:54734)
0927 18:44:08.454001 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.454012 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.454053 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.454712 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.454712 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.454714 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.454751 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.454753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.455684 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.455685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.456588 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.456591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.457451 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.457452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.457524 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.457528 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.457572 (+ 44us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.457614 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":20}
I20260927 18:44:08.458214 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.454066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.15.110.193:39719)
0927 18:44:08.454562 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.454567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.454577 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:08.454617 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.454619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.454636 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.454705 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.454803 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.455585 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.456677 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.457291 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.457979 (+ 688us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.458046 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.458069 (+ 23us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.458099 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":102,"threads_started":1}
I20260927 18:44:08.458729 23159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.454014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:39085 (local address 127.0.0.1:35664)
0927 18:44:08.454264 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.454274 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.454486 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.454725 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.454726 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.454727 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.454758 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.454759 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.456031 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.456032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.457287 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.457289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.458483 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.458484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.458545 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.458550 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.458608 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.458648 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":51,"thread_start_us":77,"threads_started":1}
I20260927 18:44:08.458752 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.454201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35664 (local address 127.15.110.195:39085)
0927 18:44:08.454359 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.454364 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.454375 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:44:08.454581 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.454583 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.454598 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.454646 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.454835 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.455907 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.457387 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.458100 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.458163 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.458211 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.458638 (+ 427us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.458679 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":87,"threads_started":1}
I20260927 18:44:08.458905 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.454057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45839 (local address 127.0.0.1:41176)
0927 18:44:08.454375 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.454386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.454432 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.454691 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.454692 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.454694 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.454742 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.454744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.456031 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.456032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.457288 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.457290 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.458689 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.458690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.458747 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.458751 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.458803 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.458845 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":68,"thread_start_us":105,"threads_started":1}
I20260927 18:44:08.458935 22799 tablet_service.cc:1467] Processing CreateTablet for tablet b4e65f524fbb4b63b18f09ad8554223c (DEFAULT_TABLE table=test-table [id=f411c0b3d8e7455f96a05e5930fd45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:08.459218 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.454290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41176 (local address 127.15.110.194:45839)
0927 18:44:08.454455 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.454459 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.454468 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:08.454520 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.454523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.454542 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.454623 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.454819 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.455919 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.457384 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.458521 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.458607 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.458656 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.459057 (+ 401us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.459104 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":69,"threads_started":1}
I20260927 18:44:08.459347 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e65f524fbb4b63b18f09ad8554223c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:08.459352 23072 tablet_service.cc:1467] Processing CreateTablet for tablet b4e65f524fbb4b63b18f09ad8554223c (DEFAULT_TABLE table=test-table [id=f411c0b3d8e7455f96a05e5930fd45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:08.459677 23072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e65f524fbb4b63b18f09ad8554223c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:08.459743 22936 tablet_service.cc:1467] Processing CreateTablet for tablet b4e65f524fbb4b63b18f09ad8554223c (DEFAULT_TABLE table=test-table [id=f411c0b3d8e7455f96a05e5930fd45a4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:08.459973 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b4e65f524fbb4b63b18f09ad8554223c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:08.462860 23164 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Bootstrap starting.
I20260927 18:44:08.463099 23165 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Bootstrap starting.
I20260927 18:44:08.463577 23164 tablet_bootstrap.cc:654] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:08.463707 23165 tablet_bootstrap.cc:654] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:08.463919 23164 log.cc:824] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:08.464495 23165 log.cc:824] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:08.464836 23166 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Bootstrap starting.
I20260927 18:44:08.465231 23164 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: No bootstrap required, opened a new log
I20260927 18:44:08.465307 23164 ts_tablet_manager.cc:1397] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:44:08.465381 23165 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: No bootstrap required, opened a new log
I20260927 18:44:08.465440 23166 tablet_bootstrap.cc:654] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:08.465448 23165 ts_tablet_manager.cc:1397] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:08.465710 23166 log.cc:824] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:08.466513 23166 tablet_bootstrap.cc:492] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: No bootstrap required, opened a new log
I20260927 18:44:08.466584 23166 ts_tablet_manager.cc:1397] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:44:08.467240 23165 raft_consensus.cc:348] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.467410 23165 raft_consensus.cc:374] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:08.467455 23165 raft_consensus.cc:729] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9511d9b369c42379543d5c449b56c29, State: Initialized, Role: FOLLOWER
I20260927 18:44:08.467381 23164 raft_consensus.cc:348] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.467558 23164 raft_consensus.cc:374] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:08.467595 23164 raft_consensus.cc:729] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965f43e36c47407aaf240304e905bde7, State: Initialized, Role: FOLLOWER
I20260927 18:44:08.467576 23165 consensus_queue.cc:260] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.467702 23164 consensus_queue.cc:260] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.467842 23165 ts_tablet_manager.cc:1428] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:44:08.467931 22866 heartbeater.cc:503] Master 127.15.110.254:45065 was elected leader, sending a full tablet report...
I20260927 18:44:08.467947 23164 ts_tablet_manager.cc:1428] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:08.468278 23003 heartbeater.cc:503] Master 127.15.110.254:45065 was elected leader, sending a full tablet report...
I20260927 18:44:08.468884 23166 raft_consensus.cc:348] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.469058 23166 raft_consensus.cc:374] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:08.469099 23166 raft_consensus.cc:729] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97383684bafb4aae8707a2f2c853affd, State: Initialized, Role: FOLLOWER
I20260927 18:44:08.469219 23166 consensus_queue.cc:260] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.469485 23166 ts_tablet_manager.cc:1428] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:08.469625 23139 heartbeater.cc:503] Master 127.15.110.254:45065 was elected leader, sending a full tablet report...
I20260927 18:44:08.487025 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.484749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56352 (local address 127.15.110.252:36267)
0927 18:44:08.484827 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.484829 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.484845 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:44:08.484869 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.484870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.484889 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.484946 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.485065 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.485549 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.486394 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.486767 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.486822 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.486872 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.486939 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.486967 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:44:08.487020 23160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.484667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56352)
0927 18:44:08.484728 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.484748 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.484792 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.484958 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.484958 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.484959 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.485006 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.485008 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.485607 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.485608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.486322 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.486324 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.486834 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.486835 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.486878 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.486881 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.486920 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.486957 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:44:08.487479 22696 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260927 18:44:08.488471 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8140d2b0fc314ba78040641c2faa5aa1. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.488579 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: LEARNER
W20260927 18:44:08.546592 23005 tablet.cc:2369] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:08.593942 23172 raft_consensus.cc:482] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:08.594128 23172 raft_consensus.cc:504] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.594537 23172 leader_election.cc:288] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719), 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
I20260927 18:44:08.598369 23163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.594966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:60263 (local address 127.15.110.194:45839)
0927 18:44:08.595014 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.595016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.595050 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:44:08.595076 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.595077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.595086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.595132 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.595278 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.596065 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.597630 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.598031 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.598099 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.598106 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.598260 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.598305 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:44:08.598369 23142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.594902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45839 (local address 127.15.110.195:60263)
0927 18:44:08.594942 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.594951 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.594981 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.595147 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.595147 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.595149 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.595222 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.595223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.596162 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.596163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.597515 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.597518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.598107 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.598108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.598169 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.598173 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.598250 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.598291 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":87}
I20260927 18:44:08.598783 22956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e65f524fbb4b63b18f09ad8554223c" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965f43e36c47407aaf240304e905bde7" is_pre_election: true
I20260927 18:44:08.598956 22956 raft_consensus.cc:2454] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 0.
I20260927 18:44:08.599030 23161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.594846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49509 (local address 127.15.110.193:39719)
0927 18:44:08.594908 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.594911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.594921 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:08.594955 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.594956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.594966 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.595059 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:08.595255 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.596046 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.598140 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.598638 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.598714 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.598722 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.598919 (+ 197us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.598968 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:44:08.599068 23144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.594769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39719 (local address 127.15.110.195:49509)
0927 18:44:08.594829 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:08.594842 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:08.594892 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:08.595068 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:08.595069 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:08.595071 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:08.595151 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.595153 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.596156 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.596158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.597515 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:08.597518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.598754 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.598755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:08.598833 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.598836 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:08.598908 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:44:08.598953 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":78}
I20260927 18:44:08.599278 23028 leader_election.cc:302] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965f43e36c47407aaf240304e905bde7, 97383684bafb4aae8707a2f2c853affd; no voters:
I20260927 18:44:08.599414 23172 raft_consensus.cc:2781] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:08.599481 23172 raft_consensus.cc:482] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:08.599505 23172 raft_consensus.cc:3037] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:08.599848 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e65f524fbb4b63b18f09ad8554223c" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9511d9b369c42379543d5c449b56c29" is_pre_election: true
I20260927 18:44:08.599998 22819 raft_consensus.cc:2454] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 0.
I20260927 18:44:08.600450 23172 raft_consensus.cc:504] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.600590 23172 leader_election.cc:288] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 election: Requested vote from peers e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719), 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839)
I20260927 18:44:08.600745 22956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e65f524fbb4b63b18f09ad8554223c" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965f43e36c47407aaf240304e905bde7"
I20260927 18:44:08.600773 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b4e65f524fbb4b63b18f09ad8554223c" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9511d9b369c42379543d5c449b56c29"
I20260927 18:44:08.600850 22956 raft_consensus.cc:3037] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:08.600870 22819 raft_consensus.cc:3037] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:08.601603 22956 raft_consensus.cc:2454] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 1.
I20260927 18:44:08.601766 23028 leader_election.cc:302] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965f43e36c47407aaf240304e905bde7, 97383684bafb4aae8707a2f2c853affd; no voters:
I20260927 18:44:08.601796 22819 raft_consensus.cc:2454] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 1.
I20260927 18:44:08.601867 23172 raft_consensus.cc:2781] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:08.602033 23172 raft_consensus.cc:686] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 1 LEADER]: Becoming Leader. State: Replica: 97383684bafb4aae8707a2f2c853affd, State: Running, Role: LEADER
I20260927 18:44:08.602171 23172 consensus_queue.cc:237] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } }
I20260927 18:44:08.603338 21901 catalog_manager.cc:5495] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd reported cstate change: term changed from 0 to 1, leader changed from <none> to 97383684bafb4aae8707a2f2c853affd (127.15.110.195). New cstate: current_term: 1 leader_uuid: "97383684bafb4aae8707a2f2c853affd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } health_report { overall_health: UNKNOWN } } }
I20260927 18:44:08.603996 22649 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: NON_VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: true } } }
I20260927 18:44:08.604108 22649 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.604604 22649 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: attempting to promote NON_VOTER 38588ae5e41e4d43baa3dfb621a7056f to VOTER
I20260927 18:44:08.604743 22649 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } }
I20260927 18:44:08.605218 22696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 LEARNER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 18:44:08.605389 23182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 9, Time since last communication: 0.000s
I20260927 18:44:08.605432 22508 raft_consensus.cc:1261] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Refusing update from remote peer 8140d2b0fc314ba78040641c2faa5aa1: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260927 18:44:08.605675 23182 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260927 18:44:08.606201 23181 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.606489 22696 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.607123 23181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.607155 22650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.607103 22508 raft_consensus.cc:2932] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.607229 23181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.607267 22650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.607339 23182 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.607398 23182 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:08.608043 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.608144 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:44:08.608304 23183 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8140d2b0fc314ba78040641c2faa5aa1" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:08.608371 23183 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:44:08.640375 23162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.637998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35680 (local address 127.15.110.195:39085)
0927 18:44:08.638063 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.638065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.638073 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:08.638108 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.638108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.638127 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.638190 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:08.638312 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.638838 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.639628 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.639827 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.639897 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.640021 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:08.640132 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:08.640204 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.640241 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.640260 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:44:08.642393 22819 raft_consensus.cc:1261] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Refusing update from remote peer 97383684bafb4aae8707a2f2c853affd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:08.642496 22956 raft_consensus.cc:1261] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Refusing update from remote peer 97383684bafb4aae8707a2f2c853affd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:08.642877 23177 consensus_queue.cc:1035] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [LEADER]: Connected to new peer: Peer: permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:08.643005 23172 consensus_queue.cc:1035] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:08.645591 23185 mvcc.cc:205] Tried to move back new op lower bound from 7334029920836161536 to 7334029920675082240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029920836161536}]
I20260927 18:44:08.645807 23186 mvcc.cc:205] Tried to move back new op lower bound from 7334029920836161536 to 7334029920675082240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029920836161536}]
I20260927 18:44:08.646555 23187 mvcc.cc:205] Tried to move back new op lower bound from 7334029920836161536 to 7334029920675082240. Current Snapshot: MvccSnapshot[applied={T|T < 7334029920836161536 or (T in {7334029920836161536})}]
I20260927 18:44:08.648723 21900 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:08.648829 21900 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:08.650701 21900 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8140d2b0fc314ba78040641c2faa5aa1: Sending DeleteTablet for 3 replicas of tablet b4e65f524fbb4b63b18f09ad8554223c
I20260927 18:44:08.651149 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e65f524fbb4b63b18f09ad8554223c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:08 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:54734
I20260927 18:44:08.651440 23195 tablet_replica.cc:330] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: stopping tablet replica
I20260927 18:44:08.651551 23195 raft_consensus.cc:2229] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:08.651543 23072 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e65f524fbb4b63b18f09ad8554223c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:08 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:35664
I20260927 18:44:08.651669 23195 raft_consensus.cc:2258] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:08.651706 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet b4e65f524fbb4b63b18f09ad8554223c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:08 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41176
I20260927 18:44:08.651777 23196 tablet_replica.cc:330] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: stopping tablet replica
I20260927 18:44:08.651865 23196 raft_consensus.cc:2229] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:08.651947 23197 tablet_replica.cc:330] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: stopping tablet replica
I20260927 18:44:08.651988 23196 raft_consensus.cc:2258] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:08.652052 23197 raft_consensus.cc:2229] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:08.652158 23197 raft_consensus.cc:2258] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:08.653121 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 21869
I20260927 18:44:08.653844 23195 ts_tablet_manager.cc:1905] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:08.654027 23196 ts_tablet_manager.cc:1905] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:08.654106 23197 ts_tablet_manager.cc:1905] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:08.655910 23195 ts_tablet_manager.cc:1918] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:08.656131 23197 ts_tablet_manager.cc:1918] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:08.656213 23197 log.cc:1196] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/b4e65f524fbb4b63b18f09ad8554223c
I20260927 18:44:08.656504 23197 ts_tablet_manager.cc:1939] T b4e65f524fbb4b63b18f09ad8554223c P 965f43e36c47407aaf240304e905bde7: Deleting consensus metadata
I20260927 18:44:08.658179 23195 log.cc:1196] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/b4e65f524fbb4b63b18f09ad8554223c
I20260927 18:44:08.658475 23196 ts_tablet_manager.cc:1918] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:08.658898 23195 ts_tablet_manager.cc:1939] T b4e65f524fbb4b63b18f09ad8554223c P e9511d9b369c42379543d5c449b56c29: Deleting consensus metadata
I20260927 18:44:08.659029 23196 log.cc:1196] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/b4e65f524fbb4b63b18f09ad8554223c
I20260927 18:44:08.659425 23196 ts_tablet_manager.cc:1939] T b4e65f524fbb4b63b18f09ad8554223c P 97383684bafb4aae8707a2f2c853affd: Deleting consensus metadata
W20260927 18:44:08.660754 23026 connection.cc:535] server connection from 127.0.0.1:35664 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:44:08.660784 22891 connection.cc:535] server connection from 127.0.0.1:41176 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:44:08.660961 22755 connection.cc:535] server connection from 127.0.0.1:54734 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:44:08.661916 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.652706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56354 (local address 127.15.110.252:36267)
0927 18:44:08.652747 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.652749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.652757 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:08.652779 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.652780 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.652791 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.652846 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.653978 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.654742 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.655944 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.656284 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.656336 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.658426 (+ 2090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.658428 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.658428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.658436 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.660470 (+ 2034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.660902 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.660904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.660904 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.660982 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.661221 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.661222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.661223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.661397 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.661507 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.661718 (+ 211us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.661808 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:08.671245 23150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.662849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56628 (local address 127.15.110.253:40299)
0927 18:44:08.662896 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.662898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.663099 (+ 201us) server_negotiation.cc:407] Connection header received
0927 18:44:08.663247 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.663249 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.663265 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.663301 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.664303 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.664858 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.666193 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.666435 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.666531 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.668060 (+ 1529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.668062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.668062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.668071 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.669863 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.670294 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.670295 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.670296 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.670354 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.670587 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.670588 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.670589 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.670721 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.670842 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.670994 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.671096 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:44:08.671675 23154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:08.663105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56362 (local address 127.15.110.252:36267)
0927 18:44:08.663147 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:08.663149 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:08.663159 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:08.663276 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:08.663277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:08.663288 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:08.663360 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:08.664288 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.665128 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.666172 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:08.666518 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:08.666595 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:08.668374 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:08.668376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:08.668376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:08.668383 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:08.670168 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.670615 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.670616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.670617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.670676 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:08.670908 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:08.670909 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:08.670910 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:08.671069 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:08.671166 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:08.671486 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:44:08.671572 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:44:09.647240 23206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:09.646910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:43439)
0927 18:44:09.647059 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:09.647152 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260927 18:44:09.647240 23207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:09.646936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:39971)
0927 18:44:09.647114 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:09.647170 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":93,"threads_started":1}
W20260927 18:44:09.647643 23139 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:45065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
W20260927 18:44:09.647881 22866 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:45065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:09.647989 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:09.647572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:51177)
0927 18:44:09.647860 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:09.647936 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":57,"threads_started":1}
W20260927 18:44:09.648135 23003 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:45065 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:10.151158 23209 raft_consensus.cc:482] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8140d2b0fc314ba78040641c2faa5aa1)
I20260927 18:44:10.151310 23209 raft_consensus.cc:504] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } }
I20260927 18:44:10.151629 23209 leader_election.cc:288] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065), 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252:36267)
I20260927 18:44:10.151858 23211 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8140d2b0fc314ba78040641c2faa5aa1)
I20260927 18:44:10.152027 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.151714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42340)
0927 18:44:10.151821 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.151872 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85,"spinlock_wait_cycles":768,"thread_start_us":48,"threads_started":1}
I20260927 18:44:10.151993 23211 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } }
W20260927 18:44:10.152163 22483 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:10.152405 23211 leader_election.cc:288] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065), be114d271da945afa411718cd0a63cdd (127.15.110.253:40299)
I20260927 18:44:10.152498 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.152413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42368)
0927 18:44:10.152449 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.152471 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 18:44:10.152554 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.152308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42352)
0927 18:44:10.152460 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.152511 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":39,"threads_started":1}
W20260927 18:44:10.152565 22483 leader_election.cc:334] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065): Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
W20260927 18:44:10.152654 22671 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:10.153129 23216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.152984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42370)
0927 18:44:10.153040 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.153082 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260927 18:44:10.153229 22671 leader_election.cc:334] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065): Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:10.155422 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.151746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36267 (local address 127.0.0.1:56376)
0927 18:44:10.151821 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.151832 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:10.151887 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:10.152181 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:10.152183 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:10.152185 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:10.152246 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.152249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.153462 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.153463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.154535 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.154537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.155219 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.155220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.155272 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.155276 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:10.155321 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 18:44:10.155375 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"thread_start_us":38,"threads_started":1}
I20260927 18:44:10.155504 23213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.151926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56376 (local address 127.15.110.252:36267)
0927 18:44:10.152015 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.152017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.152026 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:10.152049 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.152049 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.152060 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.152124 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:10.152331 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.153394 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.154641 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.155097 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.155204 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.155214 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.155345 (+ 131us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.155415 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72,"thread_start_us":48,"threads_started":1}
I20260927 18:44:10.155691 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be114d271da945afa411718cd0a63cdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "38588ae5e41e4d43baa3dfb621a7056f" is_pre_election: true
I20260927 18:44:10.155824 22696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be114d271da945afa411718cd0a63cdd in term 1.
I20260927 18:44:10.155977 22484 leader_election.cc:302] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38588ae5e41e4d43baa3dfb621a7056f, be114d271da945afa411718cd0a63cdd; no voters: 8140d2b0fc314ba78040641c2faa5aa1
I20260927 18:44:10.156016 23215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.152502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:40299 (local address 127.0.0.1:56644)
0927 18:44:10.152609 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.152619 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:10.152668 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:10.152977 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:10.152978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:10.152980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:10.153045 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.153047 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.153862 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.153863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.154897 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.154899 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.155772 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.155773 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.155835 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.155840 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:10.155898 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:44:10.155946 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":43,"threads_started":1}
I20260927 18:44:10.156080 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.152703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.15.110.253:40299)
0927 18:44:10.152815 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.152817 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.152825 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:10.152848 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.152848 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.152859 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.152928 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:10.153118 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.153786 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.154982 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.155698 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.155764 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.155771 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.155915 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.155953 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260927 18:44:10.156117 23209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260927 18:44:10.156152 23209 raft_consensus.cc:482] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8140d2b0fc314ba78040641c2faa5aa1)
I20260927 18:44:10.156170 23209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:44:10.156298 22508 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "38588ae5e41e4d43baa3dfb621a7056f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "be114d271da945afa411718cd0a63cdd" is_pre_election: true
I20260927 18:44:10.157047 23209 raft_consensus.cc:504] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } }
I20260927 18:44:10.157198 22508 raft_consensus.cc:2379] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 38588ae5e41e4d43baa3dfb621a7056f in current term 2: Already voted for candidate be114d271da945afa411718cd0a63cdd in this term.
I20260927 18:44:10.157400 22672 leader_election.cc:302] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 38588ae5e41e4d43baa3dfb621a7056f; no voters: 8140d2b0fc314ba78040641c2faa5aa1, be114d271da945afa411718cd0a63cdd
I20260927 18:44:10.157415 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.157285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42372)
0927 18:44:10.157322 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.157384 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260927 18:44:10.157437 22696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "be114d271da945afa411718cd0a63cdd" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "38588ae5e41e4d43baa3dfb621a7056f"
I20260927 18:44:10.157502 22696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:44:10.157747 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.157668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:42374)
0927 18:44:10.157701 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.157726 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260927 18:44:10.157833 22483 leader_election.cc:334] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065): Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
I20260927 18:44:10.158236 22696 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be114d271da945afa411718cd0a63cdd in term 2.
I20260927 18:44:10.158299 23211 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:44:10.158393 22484 leader_election.cc:302] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 38588ae5e41e4d43baa3dfb621a7056f, be114d271da945afa411718cd0a63cdd; no voters: 8140d2b0fc314ba78040641c2faa5aa1
I20260927 18:44:10.157199 23209 leader_election.cc:288] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [CANDIDATE]: Term 2 election: Requested vote from peers 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065), 38588ae5e41e4d43baa3dfb621a7056f (127.15.110.252:36267)
I20260927 18:44:10.158493 23209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:44:10.158533 23209 raft_consensus.cc:686] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [term 2 LEADER]: Becoming Leader. State: Replica: be114d271da945afa411718cd0a63cdd, State: Running, Role: LEADER
I20260927 18:44:10.158607 23209 consensus_queue.cc:237] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } }
I20260927 18:44:10.158943 23219 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: New leader be114d271da945afa411718cd0a63cdd. Latest consensus state: current_term: 2 leader_uuid: "be114d271da945afa411718cd0a63cdd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:10.159014 23219 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:10.159217 23221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:10.159593 23221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:10.159687 23221 catalog_manager.cc:1259] Loaded cluster ID: 702cd1b0bda9437090a603068f323ec8
I20260927 18:44:10.159720 23221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:10.159883 23221 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:10.159981 23221 catalog_manager.cc:5879] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Loaded TSK: 0
I20260927 18:44:10.160125 23221 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:10.207267 22497 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:10.208927 22497 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:10.209933 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.209808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:55062)
0927 18:44:10.209870 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.209903 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:44:10.209949 22696 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [term 2 FOLLOWER]: Refusing update from remote peer be114d271da945afa411718cd0a63cdd: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260927 18:44:10.210197 23219 consensus_queue.cc:1035] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260927 18:44:10.210263 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.210179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.0.0.1:55072)
0927 18:44:10.210209 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.210235 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260927 18:44:10.210389 22483 consensus_peers.cc:487] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd -> Peer 8140d2b0fc314ba78040641c2faa5aa1 (127.15.110.254:45065): Couldn't send request to peer 8140d2b0fc314ba78040641c2faa5aa1. Status: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260927 18:44:10.210830 23211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: New leader be114d271da945afa411718cd0a63cdd. Latest consensus state: current_term: 2 leader_uuid: "be114d271da945afa411718cd0a63cdd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:10.210940 23211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:44:10.211424 23219 sys_catalog.cc:455] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "be114d271da945afa411718cd0a63cdd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:10.211513 23219 sys_catalog.cc:458] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:10.211517 23223 mvcc.cc:205] Tried to move back new op lower bound from 7334029927257878528 to 7334029927050092544. Current Snapshot: MvccSnapshot[applied={T|T < 7334029927257878528}]
I20260927 18:44:10.211781 23211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "be114d271da945afa411718cd0a63cdd" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "8140d2b0fc314ba78040641c2faa5aa1" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 45065 } } peers { permanent_uuid: "be114d271da945afa411718cd0a63cdd" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 40299 } attrs { promote: false } } peers { permanent_uuid: "38588ae5e41e4d43baa3dfb621a7056f" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36267 } attrs { promote: false } } }
I20260927 18:44:10.211885 23211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38588ae5e41e4d43baa3dfb621a7056f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:44:10.216650 23212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:39085 (local address 127.0.0.1:36408)
0927 18:44:10.213476 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.213486 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:10.213518 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:10.213869 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:10.213870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:10.213871 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:10.213921 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.213924 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.214658 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.214659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.215680 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.215682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.216407 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.216408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.216468 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.216472 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:10.216520 (+ 48us) client_negotiation.cc:240] Negotiation successful
0927 18:44:10.216563 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260927 18:44:10.216706 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35914 (local address 127.15.110.194:45839)
0927 18:44:10.213764 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.213767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.213775 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:10.213799 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.213800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.213813 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.213902 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:10.214013 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.214770 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.215847 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.216413 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.216492 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.216500 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.216605 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.216636 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":66,"threads_started":1}
I20260927 18:44:10.216722 23210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:45839 (local address 127.0.0.1:35914)
0927 18:44:10.213381 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.213392 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:10.213438 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:10.213906 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:10.213906 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:10.213907 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:10.213932 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.213933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.214861 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.214862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.215767 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.215770 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.216477 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.216478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.216541 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.216544 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:10.216598 (+ 54us) client_negotiation.cc:240] Negotiation successful
0927 18:44:10.216627 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14,"mutex_wait_us":72}
I20260927 18:44:10.216903 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.15.110.195:39085)
0927 18:44:10.213745 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.213747 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.213756 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:10.213785 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.213786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.213797 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.213867 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:10.213991 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.214570 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.215776 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.216332 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.216414 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.216423 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.216760 (+ 337us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.216815 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260927 18:44:10.216991 22936 tablet_service.cc:1467] Processing CreateTablet for tablet 260362812aee4341a676876baefd3f06 (DEFAULT_TABLE table=test-table [id=4759a9f14a104d1f930bdf8da91262b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:10.217110 22936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260362812aee4341a676876baefd3f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:10.217489 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55380 (local address 127.15.110.193:39719)
0927 18:44:10.213942 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.213944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.213951 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:44:10.213975 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.213976 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.213986 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.214040 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:10.214245 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.214940 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.216220 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.217194 (+ 974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.217274 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.217281 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.217407 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.217437 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":86,"threads_started":1}
I20260927 18:44:10.217497 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.213448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39719 (local address 127.0.0.1:55380)
0927 18:44:10.213580 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.213589 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:10.213631 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:10.214083 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:10.214084 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:10.214086 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:10.214158 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.214160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.215030 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.215031 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.216141 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:10.216143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.217282 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.217283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:10.217336 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.217341 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:10.217403 (+ 62us) client_negotiation.cc:240] Negotiation successful
0927 18:44:10.217438 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260927 18:44:10.218451 23232 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Bootstrap starting.
I20260927 18:44:10.218528 23072 tablet_service.cc:1467] Processing CreateTablet for tablet 260362812aee4341a676876baefd3f06 (DEFAULT_TABLE table=test-table [id=4759a9f14a104d1f930bdf8da91262b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:10.218672 23072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260362812aee4341a676876baefd3f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:10.218674 22799 tablet_service.cc:1467] Processing CreateTablet for tablet 260362812aee4341a676876baefd3f06 (DEFAULT_TABLE table=test-table [id=4759a9f14a104d1f930bdf8da91262b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:10.218777 22799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260362812aee4341a676876baefd3f06. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:10.218851 23232 tablet_bootstrap.cc:654] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:10.219803 23234 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Bootstrap starting.
I20260927 18:44:10.220108 23235 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: Bootstrap starting.
I20260927 18:44:10.220201 23234 tablet_bootstrap.cc:654] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:10.220498 23235 tablet_bootstrap.cc:654] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:10.220556 23232 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: No bootstrap required, opened a new log
I20260927 18:44:10.220600 23232 ts_tablet_manager.cc:1397] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:44:10.220788 23234 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: No bootstrap required, opened a new log
I20260927 18:44:10.220803 23232 raft_consensus.cc:348] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.220856 23232 raft_consensus.cc:374] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:10.220858 23234 ts_tablet_manager.cc:1397] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:10.220875 23232 raft_consensus.cc:729] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 965f43e36c47407aaf240304e905bde7, State: Initialized, Role: FOLLOWER
I20260927 18:44:10.220933 23232 consensus_queue.cc:260] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.220964 23234 raft_consensus.cc:348] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.221035 23234 raft_consensus.cc:374] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:10.221060 23234 raft_consensus.cc:729] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97383684bafb4aae8707a2f2c853affd, State: Initialized, Role: FOLLOWER
I20260927 18:44:10.221072 23232 ts_tablet_manager.cc:1428] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:10.221110 23234 consensus_queue.cc:260] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.221256 23234 ts_tablet_manager.cc:1428] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:10.221683 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.221299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:34085)
0927 18:44:10.221586 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.221629 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":75,"threads_started":1}
I20260927 18:44:10.221779 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.221575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:34395)
0927 18:44:10.221712 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.221748 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260927 18:44:10.222141 23140 heartbeater.cc:503] Master 127.15.110.253:40299 was elected leader, sending a full tablet report...
I20260927 18:44:10.222168 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.222071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:46007)
0927 18:44:10.222108 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.222139 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260927 18:44:10.222401 23004 heartbeater.cc:503] Master 127.15.110.253:40299 was elected leader, sending a full tablet report...
I20260927 18:44:10.222764 23235 tablet_bootstrap.cc:492] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: No bootstrap required, opened a new log
I20260927 18:44:10.222823 23235 ts_tablet_manager.cc:1397] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:44:10.222930 23235 raft_consensus.cc:348] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.222994 23235 raft_consensus.cc:374] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:10.223008 23235 raft_consensus.cc:729] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9511d9b369c42379543d5c449b56c29, State: Initialized, Role: FOLLOWER
I20260927 18:44:10.223066 23235 consensus_queue.cc:260] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.223330 23235 ts_tablet_manager.cc:1428] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:10.223652 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.223395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:46629)
0927 18:44:10.223569 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.223605 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":79,"threads_started":1}
I20260927 18:44:10.223682 22867 heartbeater.cc:503] Master 127.15.110.253:40299 was elected leader, sending a full tablet report...
I20260927 18:44:10.224086 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.223849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:52371)
0927 18:44:10.224024 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.224052 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260927 18:44:10.398631 23239 raft_consensus.cc:482] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:10.398770 23239 raft_consensus.cc:504] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.398976 23239 leader_election.cc:288] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839), e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
I20260927 18:44:10.399238 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260362812aee4341a676876baefd3f06" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9511d9b369c42379543d5c449b56c29" is_pre_election: true
I20260927 18:44:10.399269 22956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260362812aee4341a676876baefd3f06" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965f43e36c47407aaf240304e905bde7" is_pre_election: true
I20260927 18:44:10.399351 22819 raft_consensus.cc:2454] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 0.
I20260927 18:44:10.399351 22956 raft_consensus.cc:2454] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 0.
I20260927 18:44:10.399531 23028 leader_election.cc:302] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97383684bafb4aae8707a2f2c853affd, e9511d9b369c42379543d5c449b56c29; no voters:
I20260927 18:44:10.399688 23239 raft_consensus.cc:2781] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:10.399755 23239 raft_consensus.cc:482] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:10.399786 23239 raft_consensus.cc:3037] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:10.400568 23239 raft_consensus.cc:504] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.400722 23239 leader_election.cc:288] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 election: Requested vote from peers 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839), e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719)
I20260927 18:44:10.400923 22956 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260362812aee4341a676876baefd3f06" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "965f43e36c47407aaf240304e905bde7"
I20260927 18:44:10.400939 22819 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260362812aee4341a676876baefd3f06" candidate_uuid: "97383684bafb4aae8707a2f2c853affd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9511d9b369c42379543d5c449b56c29"
I20260927 18:44:10.401005 22956 raft_consensus.cc:3037] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:10.401026 22819 raft_consensus.cc:3037] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:10.401703 22956 raft_consensus.cc:2454] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 1.
I20260927 18:44:10.401784 22819 raft_consensus.cc:2454] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97383684bafb4aae8707a2f2c853affd in term 1.
I20260927 18:44:10.401889 23028 leader_election.cc:302] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 965f43e36c47407aaf240304e905bde7, 97383684bafb4aae8707a2f2c853affd; no voters:
I20260927 18:44:10.402012 23239 raft_consensus.cc:2781] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:10.402207 23239 raft_consensus.cc:686] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 1 LEADER]: Becoming Leader. State: Replica: 97383684bafb4aae8707a2f2c853affd, State: Running, Role: LEADER
I20260927 18:44:10.402256 23239 consensus_queue.cc:237] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } }
I20260927 18:44:10.402737 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.402573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:47445)
0927 18:44:10.402666 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.402701 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260927 18:44:10.402967 22497 catalog_manager.cc:5495] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd reported cstate change: term changed from 0 to 1, leader changed from <none> to 97383684bafb4aae8707a2f2c853affd (127.15.110.195). New cstate: current_term: 1 leader_uuid: "97383684bafb4aae8707a2f2c853affd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "97383684bafb4aae8707a2f2c853affd" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 39085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 } health_report { overall_health: UNKNOWN } } }
W20260927 18:44:10.414335 22868 tablet.cc:2369] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:10.430222 23141 tablet.cc:2369] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:10.443706 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.441401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36412 (local address 127.15.110.195:39085)
0927 18:44:10.441461 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:10.441463 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:10.441478 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:44:10.441500 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:10.441501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:10.441509 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:10.441564 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:10.441708 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.442185 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.442978 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:10.443266 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:10.443337 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:10.443494 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:10.443521 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:10.443561 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:10.443622 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 18:44:10.443642 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:44:10.444634 22819 raft_consensus.cc:1261] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Refusing update from remote peer 97383684bafb4aae8707a2f2c853affd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:10.444715 22956 raft_consensus.cc:1261] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Refusing update from remote peer 97383684bafb4aae8707a2f2c853affd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:10.444886 23239 consensus_queue.cc:1035] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9511d9b369c42379543d5c449b56c29" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:10.445031 23244 consensus_queue.cc:1035] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [LEADER]: Connected to new peer: Peer: permanent_uuid: "965f43e36c47407aaf240304e905bde7" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 45839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:10.445868 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.445760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:35331)
0927 18:44:10.445803 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.445834 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 18:44:10.445924 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.445828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:59225)
0927 18:44:10.445870 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.445895 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 18:44:10.447274 23240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.447153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.194:53703)
0927 18:44:10.447213 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.447246 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:44:10.447335 23241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.447241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.195:40521)
0927 18:44:10.447278 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.447307 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260927 18:44:10.447654 23243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.447535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:45065 (local address 127.15.110.193:49979)
0927 18:44:10.447606 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.447629 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:45065: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260927 18:44:10.448824 22498 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:10.448925 22498 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56628:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:10.450222 22498 catalog_manager.cc:5782] T 00000000000000000000000000000000 P be114d271da945afa411718cd0a63cdd: Sending DeleteTablet for 3 replicas of tablet 260362812aee4341a676876baefd3f06
I20260927 18:44:10.450474 22936 tablet_service.cc:1514] Processing DeleteTablet for tablet 260362812aee4341a676876baefd3f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:10 UTC) from {username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:35914
I20260927 18:44:10.450508 22799 tablet_service.cc:1514] Processing DeleteTablet for tablet 260362812aee4341a676876baefd3f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:10 UTC) from {username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:55380
I20260927 18:44:10.450562 23072 tablet_service.cc:1514] Processing DeleteTablet for tablet 260362812aee4341a676876baefd3f06 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:10 UTC) from {username='oryx', principal='oryx/127.15.110.253@KRBTEST.COM'} at 127.0.0.1:36408
I20260927 18:44:10.450794 23257 tablet_replica.cc:330] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: stopping tablet replica
I20260927 18:44:10.450861 23257 raft_consensus.cc:2229] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:10.450884 23255 tablet_replica.cc:330] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29: stopping tablet replica
I20260927 18:44:10.450928 23257 raft_consensus.cc:2258] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:10.450944 23255 raft_consensus.cc:2229] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:10.450953 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22646
I20260927 18:44:10.450990 23255 raft_consensus.cc:2258] T 260362812aee4341a676876baefd3f06 P e9511d9b369c42379543d5c449b56c29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:10.451529 23256 tablet_replica.cc:330] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: stopping tablet replica
I20260927 18:44:10.451583 23256 raft_consensus.cc:2229] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:10.451627 23256 raft_consensus.cc:2258] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:10.452172 23256 ts_tablet_manager.cc:1905] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:10.453812 23256 ts_tablet_manager.cc:1918] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:10.453826 23257 ts_tablet_manager.cc:1905] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:10.453881 23256 log.cc:1196] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/260362812aee4341a676876baefd3f06
I20260927 18:44:10.454133 23256 ts_tablet_manager.cc:1939] T 260362812aee4341a676876baefd3f06 P 965f43e36c47407aaf240304e905bde7: Deleting consensus metadata
I20260927 18:44:10.454483 22485 catalog_manager.cc:4841] TS 965f43e36c47407aaf240304e905bde7 (127.15.110.194:45839): tablet 260362812aee4341a676876baefd3f06 (table test-table [id=4759a9f14a104d1f930bdf8da91262b3]) successfully deleted
I20260927 18:44:10.455710 23257 ts_tablet_manager.cc:1918] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:10.455780 23257 log.cc:1196] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/260362812aee4341a676876baefd3f06
I20260927 18:44:10.456012 23257 ts_tablet_manager.cc:1939] T 260362812aee4341a676876baefd3f06 P 97383684bafb4aae8707a2f2c853affd: Deleting consensus metadata
I20260927 18:44:10.456437 22482 catalog_manager.cc:4841] TS 97383684bafb4aae8707a2f2c853affd (127.15.110.195:39085): tablet 260362812aee4341a676876baefd3f06 (table test-table [id=4759a9f14a104d1f930bdf8da91262b3]) successfully deleted
I20260927 18:44:10.457084 23228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:10.456957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39719 (local address 127.0.0.1:55394)
0927 18:44:10.457028 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:10.457058 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39719: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260927 18:44:10.457170 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22873
W20260927 18:44:10.457276 22485 catalog_manager.cc:4584] TS e9511d9b369c42379543d5c449b56c29 (127.15.110.193:39719): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 260362812aee4341a676876baefd3f06: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39719: connect: Connection refused (error 111)
I20260927 18:44:10.464936 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23009
I20260927 18:44:10.470932 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22365
I20260927 18:44:10.476260 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 22552
2026-09-27T18:44:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (4865 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:10 dist-test-slave-vs05 krb5kdc[23260](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:10 dist-test-slave-vs05 krb5kdc[23260](info): set up 2 sockets
Sep 27 18:44:10 dist-test-slave-vs05 krb5kdc[23260](info): commencing operation
krb5kdc: starting...
W20260927 18:44:11.599567 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:11 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:11Z Disabled control of system clock
WARNING: no policy specified for oryx/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab.
Entry for principal oryx/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab.
I20260927 18:44:11.722154 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46681
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab
--principal=oryx/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:11.801601 23273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:11.801797 23273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:11.801818 23273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:11.803018 23273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:11.803061 23273 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260927 18:44:11.803076 23273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:11.803088 23273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:11.803100 23273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:11.803112 23273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:11.804704 23273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46681
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.254.keytab
--principal=oryx/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:11.804960 23273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:11.805198 23273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:11.810006 23278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:11.810122 23279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:11.810007 23281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:11.810230 23273 server_base.cc:1034] running on GCE node
I20260927 18:44:11.810576 23273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:11.810861 23273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:11.812012 23273 hybrid_clock.cc:648] HybridClock initialized: now 1790534651811989 us; error 44 us; skew 500 ppm
Sep 27 18:44:11 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:11.814476 23273 init.cc:377] Logged in from keytab as oryx/127.15.110.254@KRBTEST.COM (short username oryx)
I20260927 18:44:11.814937 23273 webserver.cc:460] Webserver started at http://127.15.110.254:36449/ using document root <none> and password file <none>
I20260927 18:44:11.815165 23273 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:11.815284 23273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:11.815442 23273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:11.816516 23273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "05dc5caaca544024b1cef7319ed43170"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "60defe6c5b590a0f869635ab41e9c330"
server_key_iv: "0b9faf1498bd87bdab5ed7baba15f7a7"
server_key_version: "encryptionkey@0"
I20260927 18:44:11.816893 23273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "05dc5caaca544024b1cef7319ed43170"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "60defe6c5b590a0f869635ab41e9c330"
server_key_iv: "0b9faf1498bd87bdab5ed7baba15f7a7"
server_key_version: "encryptionkey@0"
I20260927 18:44:11.818188 23273 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:11.818941 23288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:11.819108 23273 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:11.819233 23273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "05dc5caaca544024b1cef7319ed43170"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "60defe6c5b590a0f869635ab41e9c330"
server_key_iv: "0b9faf1498bd87bdab5ed7baba15f7a7"
server_key_version: "encryptionkey@0"
I20260927 18:44:11.819324 23273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:11.837217 23273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:11.839121 23273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:11.839309 23273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:11.843811 23273 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46681
I20260927 18:44:11.843918 23350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46681 every 8 connection(s)
I20260927 18:44:11.844228 23273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:11.844903 23351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:11.847083 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23273
I20260927 18:44:11.847131 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Bootstrap starting.
I20260927 18:44:11.847239 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:11.847416 15803 external_mini_cluster.cc:1428] Setting key 4af4d44671732025acbc1f816bc3e91a
I20260927 18:44:11.847887 23351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:11.848228 23351 log.cc:824] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:11.849010 23351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: No bootstrap required, opened a new log
I20260927 18:44:11.851308 23351 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } }
I20260927 18:44:11.851500 23351 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:11.851541 23351 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05dc5caaca544024b1cef7319ed43170, State: Initialized, Role: FOLLOWER
I20260927 18:44:11.851704 23351 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } }
I20260927 18:44:11.851790 23351 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:11.851835 23351 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:11.851893 23351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:11.852517 23351 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } }
I20260927 18:44:11.852653 23351 leader_election.cc:302] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05dc5caaca544024b1cef7319ed43170; no voters:
I20260927 18:44:11.852813 23351 leader_election.cc:288] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:11.852865 23356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:11.853073 23351 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:11.853058 23356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [term 1 LEADER]: Becoming Leader. State: Replica: 05dc5caaca544024b1cef7319ed43170, State: Running, Role: LEADER
I20260927 18:44:11.853215 23356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } }
I20260927 18:44:11.853559 23356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05dc5caaca544024b1cef7319ed43170. Latest consensus state: current_term: 1 leader_uuid: "05dc5caaca544024b1cef7319ed43170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } } }
I20260927 18:44:11.853626 23356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:11.853773 23357 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05dc5caaca544024b1cef7319ed43170" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05dc5caaca544024b1cef7319ed43170" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46681 } } }
I20260927 18:44:11.853829 23357 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:11.854063 23363 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:11.854547 23363 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 18:44:11 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:11.856288 23363 catalog_manager.cc:1347] Generated new cluster ID: eacaea6b41be45c6a779647969d9dad2
I20260927 18:44:11.856343 23363 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:11.858922 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:11.847942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56364 (local address 127.15.110.254:46681)
0927 18:44:11.848108 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:11.848116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:11.848142 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:44:11.848624 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:11.848631 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:11.848680 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:11.848797 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:11.849583 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:11.850433 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:11.851595 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:11.852185 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:11.852275 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:11.855397 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:11.855401 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:11.855401 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:11.855412 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:11.857333 (+ 1921us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:11.857713 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:11.857716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:11.857717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:11.857816 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:11.858051 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:11.858052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:11.858053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:11.858290 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:11.858472 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:11.858577 (+ 105us) server_negotiation.cc:299] Negotiation successful
0927 18:44:11.858718 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260927 18:44:11.865492 23363 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:11.866222 23363 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:11.871013 23363 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Generated new TSK 0
I20260927 18:44:11.871217 23363 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
I20260927 18:44:11.901675 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46681
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:11.986366 23377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:11.986568 23377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:11.986594 23377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:11.987710 23377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:11.987783 23377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:11.987819 23377 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:44:11.987854 23377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:11.989392 23377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:11.989576 23377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:11.989746 23377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:11.994570 23385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:11.994658 23382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:11.994588 23383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:11.994684 23377 server_base.cc:1034] running on GCE node
I20260927 18:44:11.994999 23377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:11.995318 23377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:11.996480 23377 hybrid_clock.cc:648] HybridClock initialized: now 1790534651996473 us; error 35 us; skew 500 ppm
Sep 27 18:44:11 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:11.998996 23377 init.cc:377] Logged in from keytab as oryx/127.15.110.193@KRBTEST.COM (short username oryx)
I20260927 18:44:11.999491 23377 webserver.cc:460] Webserver started at http://127.15.110.193:33919/ using document root <none> and password file <none>
I20260927 18:44:11.999725 23377 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:11.999794 23377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:11.999944 23377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:12.000972 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "3a1cdcf228df4d5088f8e674aab76937"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "5392987ebede8c1c02fb65943c9ac8e1"
server_key_iv: "4ec24bc9f25d4b77f3a5cdc241eb4504"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.001338 23377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "3a1cdcf228df4d5088f8e674aab76937"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "5392987ebede8c1c02fb65943c9ac8e1"
server_key_iv: "4ec24bc9f25d4b77f3a5cdc241eb4504"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.002679 23377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:12.003687 23392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.003855 23377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:12.003953 23377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "3a1cdcf228df4d5088f8e674aab76937"
format_stamp: "Formatted at 2026-09-27 18:44:11 on dist-test-slave-vs05"
server_key: "5392987ebede8c1c02fb65943c9ac8e1"
server_key_iv: "4ec24bc9f25d4b77f3a5cdc241eb4504"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.004030 23377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:12.016532 23377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:12.018446 23377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:12.018615 23377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:12.019279 23377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:12.019333 23377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.019379 23377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:12.019407 23377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.026826 23377 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:36193
I20260927 18:44:12.026875 23507 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:36193 every 8 connection(s)
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:12.027252 23377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:12.030246 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.019655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:33739 (local address 127.15.110.254:46681)
0927 18:44:12.019722 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.019725 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.019957 (+ 232us) server_negotiation.cc:407] Connection header received
0927 18:44:12.020718 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.020719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.020730 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.020810 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.021781 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.022662 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.024430 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.024791 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.024868 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.027433 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.027435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.027435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.027443 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.028896 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.029299 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.029300 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.029301 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.029361 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.029610 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.029611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.029612 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.029727 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.029804 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.030066 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.030155 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:44:12.030396 23401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.019632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.193:33739)
0927 18:44:12.019798 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.019848 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.020592 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.020916 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.020921 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.021325 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.021646 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.021654 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.022763 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.022764 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.024306 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.024309 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.024881 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.024883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.025039 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.025713 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.025724 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.027340 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.029004 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.029011 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.029014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.029225 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.029429 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.029430 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.029431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.029545 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.029794 (+ 249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.029800 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.029933 (+ 133us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.030032 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.030180 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260927 18:44:12.030989 23508 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46681
I20260927 18:44:12.031068 23508 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:12.031277 23508 heartbeater.cc:511] Master 127.15.110.254:46681 requested a full tablet report, sending...
I20260927 18:44:12.031822 23304 ts_manager.cc:194] Registered new tserver with Master: 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193)
I20260927 18:44:12.032696 23304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:33739
I20260927 18:44:12.036756 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23377
I20260927 18:44:12.036875 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:12.037081 15803 external_mini_cluster.cc:1428] Setting key 79b8b25494f4a63628d14fbe16b0e2cb
I20260927 18:44:12.037601 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.032070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.193:36231)
0927 18:44:12.032205 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.032215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.032269 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.032378 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.032379 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.032518 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.032570 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.032571 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.033290 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.033291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.033907 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.033909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.034231 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.034232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.034291 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.034749 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.034754 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.035062 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.036430 (+ 1368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.036430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.036431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.036679 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.036927 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.036928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.036929 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.037006 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.037304 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.037305 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.037356 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.037439 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.037531 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":74,"threads_started":1}
I20260927 18:44:12.037949 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.032135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:36231 (local address 127.15.110.254:46681)
0927 18:44:12.032173 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.032175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.032279 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:44:12.032301 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.032302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.032311 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.032376 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.032653 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.033203 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.033977 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.034150 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.034198 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.035132 (+ 934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.035133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.035133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.035137 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.036357 (+ 1220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.036751 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.036752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.036752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.036811 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.037092 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.037094 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.037094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.037225 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.037297 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.037770 (+ 473us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.037855 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
I20260927 18:44:12.062455 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46681
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:12.145228 23517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:12.145429 23517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:12.145455 23517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:12.146600 23517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:12.146636 23517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:12.146657 23517 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:44:12.146674 23517 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:12.148125 23517 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:12.148303 23517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:12.148468 23517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:12.153432 23523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:12.153548 23517 server_base.cc:1034] running on GCE node
W20260927 18:44:12.153416 23522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:12.153403 23525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:12.154017 23517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:12.154279 23517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:12.155416 23517 hybrid_clock.cc:648] HybridClock initialized: now 1790534652155393 us; error 32 us; skew 500 ppm
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:12.157828 23517 init.cc:377] Logged in from keytab as oryx/127.15.110.194@KRBTEST.COM (short username oryx)
I20260927 18:44:12.158259 23517 webserver.cc:460] Webserver started at http://127.15.110.194:41045/ using document root <none> and password file <none>
I20260927 18:44:12.158480 23517 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:12.158524 23517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:12.158668 23517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:12.159839 23517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "131b6f87839443cc90313c3a1b7146f2"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "28a33e6a7f713faecddf3d7f7db0e867"
server_key_iv: "d8ec3cab059dfe3dd1ebf1088fddea74"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.160193 23517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "131b6f87839443cc90313c3a1b7146f2"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "28a33e6a7f713faecddf3d7f7db0e867"
server_key_iv: "d8ec3cab059dfe3dd1ebf1088fddea74"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.161497 23517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:12.162222 23532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.162403 23517 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:12.162490 23517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "131b6f87839443cc90313c3a1b7146f2"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "28a33e6a7f713faecddf3d7f7db0e867"
server_key_iv: "d8ec3cab059dfe3dd1ebf1088fddea74"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.162544 23517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:12.174440 23517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:12.176769 23517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:12.176884 23517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:12.177364 23517 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:12.177407 23517 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.177431 23517 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:12.177443 23517 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:12.184391 23517 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:37391
I20260927 18:44:12.184458 23647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:37391 every 8 connection(s)
I20260927 18:44:12.184753 23517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:12.187006 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23517
I20260927 18:44:12.187134 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:12.187350 15803 external_mini_cluster.cc:1428] Setting key 02891440555b1584e7f51755579ac24d
I20260927 18:44:12.188066 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.177922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:49491 (local address 127.15.110.254:46681)
0927 18:44:12.178000 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.178002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.178151 (+ 149us) server_negotiation.cc:407] Connection header received
0927 18:44:12.178659 (+ 508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.178661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.178673 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.178739 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.179713 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.180589 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.181616 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.181961 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.182026 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.184897 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.184899 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.184899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.184907 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.186497 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.186944 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.186946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.186946 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.187014 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.187299 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.187299 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.187300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.187440 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.187498 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.187873 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.187957 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:44:12.188203 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.177847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.194:49491)
0927 18:44:12.178020 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.178061 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.178561 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.178813 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.178817 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.179260 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.179566 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.179574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.180693 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.180694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.181506 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.181508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.182043 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.182044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.182198 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.183244 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.183258 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.184814 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.186603 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.186610 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.186613 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.186865 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.187082 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.187083 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.187083 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.187237 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.187523 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.187529 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.187717 (+ 188us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.187816 (+ 99us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.187969 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":84,"threads_started":1}
I20260927 18:44:12.189939 23648 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46681
I20260927 18:44:12.190047 23648 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:12.190235 23648 heartbeater.cc:511] Master 127.15.110.254:46681 requested a full tablet report, sending...
I20260927 18:44:12.190619 23304 ts_manager.cc:194] Registered new tserver with Master: 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391)
I20260927 18:44:12.191099 23304 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:49491
WARNING: no policy specified for oryx/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.195@KRBTEST.COM" created.
I20260927 18:44:12.201632 23655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.195391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.194:35329)
0927 18:44:12.195525 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.195536 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.195603 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.195708 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.195709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.195862 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.195925 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.195927 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.196643 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.196644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.197167 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.197168 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.197493 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.197494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.197545 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.198021 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.198026 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.198435 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.200612 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.200613 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.200613 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.200774 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.200993 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.200994 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.200994 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.201045 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.201382 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.201383 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.201429 (+ 46us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.201480 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.201554 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":77,"threads_started":1}
I20260927 18:44:12.201781 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.195460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:35329 (local address 127.15.110.254:46681)
0927 18:44:12.195500 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.195502 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.195589 (+ 87us) server_negotiation.cc:407] Connection header received
0927 18:44:12.195644 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.195645 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.195653 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.195698 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.195984 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.196561 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.197230 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.197417 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.197482 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.198525 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.198527 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.198528 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.198537 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.200534 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.200859 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.200860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.200861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.200924 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.201106 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.201107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.201107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.201317 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.201390 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.201534 (+ 144us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.201639 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
I20260927 18:44:12.213609 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46681
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:12.296347 23657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:12.296589 23657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:12.296629 23657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:12.297793 23657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:12.297865 23657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:12.297900 23657 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260927 18:44:12.297941 23657 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:12.299477 23657 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.15.110.254:46681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:12.299752 23657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:12.299990 23657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:12.305199 23665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:12.305244 23657 server_base.cc:1034] running on GCE node
W20260927 18:44:12.305209 23663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:12.305208 23662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:12.305743 23657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:12.306010 23657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:12.307166 23657 hybrid_clock.cc:648] HybridClock initialized: now 1790534652307158 us; error 31 us; skew 500 ppm
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:12.309677 23657 init.cc:377] Logged in from keytab as oryx/127.15.110.195@KRBTEST.COM (short username oryx)
I20260927 18:44:12.310070 23657 webserver.cc:460] Webserver started at http://127.15.110.195:39553/ using document root <none> and password file <none>
I20260927 18:44:12.310276 23657 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:12.310321 23657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:12.310410 23657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:12.311529 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "71cfa157b02014503f1fc2aef3dcbb5d"
server_key_iv: "ca2a380a328d6c78135d752b6235b196"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.311836 23657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "71cfa157b02014503f1fc2aef3dcbb5d"
server_key_iv: "ca2a380a328d6c78135d752b6235b196"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.313117 23657 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:12.313856 23672 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.314062 23657 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:12.314141 23657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
format_stamp: "Formatted at 2026-09-27 18:44:12 on dist-test-slave-vs05"
server_key: "71cfa157b02014503f1fc2aef3dcbb5d"
server_key_iv: "ca2a380a328d6c78135d752b6235b196"
server_key_version: "encryptionkey@0"
I20260927 18:44:12.314194 23657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:12.326824 23657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:12.329214 23657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:12.329329 23657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:12.329802 23657 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:12.329840 23657 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.329865 23657 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:12.329905 23657 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:12.337106 23657 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40017
I20260927 18:44:12.337244 23787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40017 every 8 connection(s)
I20260927 18:44:12.337515 23657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:12.338011 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23657
I20260927 18:44:12.338169 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:12.338413 15803 external_mini_cluster.cc:1428] Setting key 5be58b7d9a0a3e7a1535e884d9f69177
I20260927 18:44:12.340680 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.330385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:57501 (local address 127.15.110.254:46681)
0927 18:44:12.330459 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.330462 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.330704 (+ 242us) server_negotiation.cc:407] Connection header received
0927 18:44:12.331482 (+ 778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.331483 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.331551 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.331622 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.332278 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.333106 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.334749 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.335075 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.335162 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.337647 (+ 2485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.337649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.337649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.337657 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.339338 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.339757 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.339758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.339758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.339810 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.340032 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.340034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.340034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.340164 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.340229 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.340552 (+ 323us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.340619 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:44:12.340759 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.330340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.195:57501)
0927 18:44:12.330543 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.330595 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.331379 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.331705 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.331709 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.331966 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.332168 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.332172 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.333189 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.333190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.334625 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.334628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.335172 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.335173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.335344 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.335936 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.335947 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.337538 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.339437 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.339444 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.339446 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.339691 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.339857 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.339857 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.339857 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.339943 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.340240 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.340245 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.340395 (+ 150us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.340483 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.340602 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":105,"threads_started":1}
I20260927 18:44:12.341254 23788 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46681
I20260927 18:44:12.341329 23788 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:12.341511 23788 heartbeater.cc:511] Master 127.15.110.254:46681 requested a full tablet report, sending...
I20260927 18:44:12.341905 23305 ts_manager.cc:194] Registered new tserver with Master: 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:12.342319 23305 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:57501
I20260927 18:44:12.347687 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.342354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:58403 (local address 127.15.110.254:46681)
0927 18:44:12.342395 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.342397 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.342653 (+ 256us) server_negotiation.cc:407] Connection header received
0927 18:44:12.342677 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.342678 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.342688 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.342751 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.343061 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.343562 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.344211 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.344382 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.344429 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.345205 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.345207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.345207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.345213 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.346547 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.346847 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.346848 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.346848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.346895 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.347055 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.347055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.347056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.347155 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.347278 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.347565 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.347623 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:44:12.347697 23794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.342256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.15.110.195:58403)
0927 18:44:12.342422 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.342433 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.342503 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.342761 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.342762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.342914 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.342989 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.342990 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.343644 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.343645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.344145 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.344147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.344442 (+ 295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.344443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.344495 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.344862 (+ 367us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.344866 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.345136 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.346629 (+ 1493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.346630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.346630 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.346785 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.346951 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.346952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.346952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.347007 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.347370 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.347371 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.347418 (+ 47us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.347536 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.347623 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":69,"threads_started":1}
I20260927 18:44:12.350309 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:44:12.372063 23354 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:12.366384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56374 (local address 127.15.110.254:46681)
0927 18:44:12.366455 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.366461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.366470 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:12.366492 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.366493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.366502 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.366559 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.367111 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.367685 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.368444 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.368648 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.368731 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.371928 (+ 3197us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56374: BlockingRecv error: recv got EOF from 127.0.0.1:56374 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534652, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:12.379568 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.372873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.15.110.254:46681)
0927 18:44:12.372932 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.372934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.373031 (+ 97us) server_negotiation.cc:407] Connection header received
0927 18:44:12.373108 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.373109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.373119 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.373186 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.373719 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.374203 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.374890 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.375091 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.375166 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.377305 (+ 2139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.377306 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.377306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.377313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.378646 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.378942 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.378943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.378943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.378991 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.379146 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.379146 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.379146 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.379286 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.379359 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.379450 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.379504 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:44:12.381152 23305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:12.382740 23305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:12.389338 23810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.0.0.1:36660)
0927 18:44:12.385474 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.385481 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.385535 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.385871 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.385872 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.385874 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.385958 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.385960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.386810 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.386812 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.388015 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.388018 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.389000 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389002 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.389093 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389107 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.389191 (+ 84us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.389254 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"mutex_wait_us":7,"spinlock_wait_cycles":768,"thread_start_us":90,"threads_started":1}
I20260927 18:44:12.389467 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41444 (local address 127.15.110.195:40017)
0927 18:44:12.385659 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.385666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.385736 (+ 70us) server_negotiation.cc:407] Connection header received
0927 18:44:12.385781 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.385783 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.385798 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.385868 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.386053 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.387200 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.388399 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.389090 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389217 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389270 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.389336 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.389372 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":94,"threads_started":1}
I20260927 18:44:12.389621 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36660 (local address 127.15.110.193:36193)
0927 18:44:12.385606 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.385612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.385645 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:44:12.385689 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.385691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.385706 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.385783 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.386023 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.386723 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.388105 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.388911 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389011 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389071 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.389467 (+ 396us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.389512 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":80,"threads_started":1}
I20260927 18:44:12.389801 23813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40017 (local address 127.0.0.1:41444)
0927 18:44:12.385578 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.385590 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.385647 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.385933 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.385934 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.385936 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.385977 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.385979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.387282 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.387283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.388295 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.388297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.389200 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.389259 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389264 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.389323 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.389363 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"mutex_wait_us":38,"thread_start_us":69,"threads_started":1}
I20260927 18:44:12.390035 23811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:37391 (local address 127.0.0.1:59852)
0927 18:44:12.385269 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.385310 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.385375 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.385864 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.385867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.385869 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.385948 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.385950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.387328 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.387329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.388318 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.388320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.389368 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.389579 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389585 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.389666 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.389974 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":3,"thread_start_us":61,"threads_started":1}
I20260927 18:44:12.390360 23442 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb (DEFAULT_TABLE table=test-table [id=2d6f478e2b714e0e97116bea31e2e2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.390471 23722 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb (DEFAULT_TABLE table=test-table [id=2d6f478e2b714e0e97116bea31e2e2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.390744 23442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4efbc6769c4944a2d1dc647a140dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.390769 23722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4efbc6769c4944a2d1dc647a140dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.391342 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.385402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59852 (local address 127.15.110.194:37391)
0927 18:44:12.385676 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.385680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.385689 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:12.385729 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.385731 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.385750 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.386029 (+ 279us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.386090 (+ 61us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.387167 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.388422 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.389186 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.389290 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.389344 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.389790 (+ 446us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.389838 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":90,"threads_started":1}
I20260927 18:44:12.394449 23816 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Bootstrap starting.
I20260927 18:44:12.394491 23817 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Bootstrap starting.
I20260927 18:44:12.395161 23816 tablet_bootstrap.cc:654] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.395164 23817 tablet_bootstrap.cc:654] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.395288 23582 tablet_service.cc:1467] Processing CreateTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb (DEFAULT_TABLE table=test-table [id=2d6f478e2b714e0e97116bea31e2e2de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.395533 23582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f4efbc6769c4944a2d1dc647a140dcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.395530 23816 log.cc:824] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:12.395530 23817 log.cc:824] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:12.396289 23817 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: No bootstrap required, opened a new log
I20260927 18:44:12.396375 23817 ts_tablet_manager.cc:1397] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:44:12.396399 23816 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: No bootstrap required, opened a new log
I20260927 18:44:12.396463 23816 ts_tablet_manager.cc:1397] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:44:12.397864 23817 raft_consensus.cc:348] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.398015 23817 raft_consensus.cc:374] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.398056 23817 raft_consensus.cc:729] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.398203 23817 consensus_queue.cc:260] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.398488 23817 ts_tablet_manager.cc:1428] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:44:12.398617 23508 heartbeater.cc:503] Master 127.15.110.254:46681 was elected leader, sending a full tablet report...
I20260927 18:44:12.398581 23816 raft_consensus.cc:348] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.398747 23816 raft_consensus.cc:374] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.398787 23816 raft_consensus.cc:729] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.398895 23816 consensus_queue.cc:260] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.399147 23816 ts_tablet_manager.cc:1428] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:44:12.399194 23788 heartbeater.cc:503] Master 127.15.110.254:46681 was elected leader, sending a full tablet report...
I20260927 18:44:12.400341 23822 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Bootstrap starting.
I20260927 18:44:12.400987 23822 tablet_bootstrap.cc:654] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.401276 23822 log.cc:824] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:12.402048 23822 tablet_bootstrap.cc:492] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: No bootstrap required, opened a new log
I20260927 18:44:12.402107 23822 ts_tablet_manager.cc:1397] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:44:12.403563 23822 raft_consensus.cc:348] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.403688 23822 raft_consensus.cc:374] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.403710 23822 raft_consensus.cc:729] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.403841 23822 consensus_queue.cc:260] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.404070 23822 ts_tablet_manager.cc:1428] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:12.404104 23648 heartbeater.cc:503] Master 127.15.110.254:46681 was elected leader, sending a full tablet report...
W20260927 18:44:12.435384 23649 tablet.cc:2369] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:12.528029 23509 tablet.cc:2369] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:12.588874 23789 tablet.cc:2369] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:12.670979 23821 raft_consensus.cc:482] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:12.671106 23821 raft_consensus.cc:504] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.671602 23821 leader_election.cc:288] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391)
I20260927 18:44:12.674531 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.671944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:47927 (local address 127.15.110.193:36193)
0927 18:44:12.672019 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.672021 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.672029 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:12.672057 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.672059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.672069 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.672177 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.672381 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.672857 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.673809 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.674173 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.674259 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.674267 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.674434 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.674470 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:44:12.674543 23681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.671798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.15.110.195:47927)
0927 18:44:12.671860 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.671875 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.671925 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.672215 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.672216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.672218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.672315 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.672318 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.672945 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.672947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.673711 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.673713 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.674257 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.674258 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.674350 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.674353 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.674416 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.674455 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:44:12.674604 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.671849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:39187 (local address 127.15.110.194:37391)
0927 18:44:12.671912 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.671914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.672286 (+ 372us) server_negotiation.cc:407] Connection header received
0927 18:44:12.672321 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.672322 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.672332 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.672383 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.672539 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.673017 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.673897 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.674261 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.674341 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.674349 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.674501 (+ 152us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.674535 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:44:12.674863 23825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.671805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:37391 (local address 127.15.110.195:39187)
0927 18:44:12.672202 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.672212 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.672273 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.672384 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.672385 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.672387 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.672466 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.672468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.673079 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.673081 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.673804 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.673805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.674351 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.674352 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.674427 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.674430 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.674494 (+ 64us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.674532 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"mutex_wait_us":170,"thread_start_us":165,"threads_started":1}
I20260927 18:44:12.674973 23462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4efbc6769c4944a2d1dc647a140dcb" candidate_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937" is_pre_election: true
I20260927 18:44:12.675127 23462 raft_consensus.cc:2454] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c40d416a7ba44c0b0f231a70eaea64d in term 0.
I20260927 18:44:12.675299 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4efbc6769c4944a2d1dc647a140dcb" candidate_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2" is_pre_election: true
I20260927 18:44:12.675391 23675 leader_election.cc:302] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1cdcf228df4d5088f8e674aab76937, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:12.675436 23602 raft_consensus.cc:2454] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7c40d416a7ba44c0b0f231a70eaea64d in term 0.
I20260927 18:44:12.675523 23821 raft_consensus.cc:2781] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:12.675631 23821 raft_consensus.cc:482] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:12.675660 23821 raft_consensus.cc:3037] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.676594 23821 raft_consensus.cc:504] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.676740 23821 leader_election.cc:288] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [CANDIDATE]: Term 1 election: Requested vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391)
I20260927 18:44:12.676851 23462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4efbc6769c4944a2d1dc647a140dcb" candidate_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937"
I20260927 18:44:12.676877 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f4efbc6769c4944a2d1dc647a140dcb" candidate_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2"
I20260927 18:44:12.676951 23602 raft_consensus.cc:3037] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.676947 23462 raft_consensus.cc:3037] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.677594 23462 raft_consensus.cc:2454] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c40d416a7ba44c0b0f231a70eaea64d in term 1.
I20260927 18:44:12.677619 23602 raft_consensus.cc:2454] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7c40d416a7ba44c0b0f231a70eaea64d in term 1.
I20260927 18:44:12.677783 23676 leader_election.cc:302] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:12.677902 23821 raft_consensus.cc:2781] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:12.678102 23821 raft_consensus.cc:686] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 LEADER]: Becoming Leader. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Running, Role: LEADER
I20260927 18:44:12.678212 23821 consensus_queue.cc:237] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.679057 23305 catalog_manager.cc:5495] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195). New cstate: current_term: 1 leader_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } health_report { overall_health: HEALTHY } } }
I20260927 18:44:12.746214 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.744078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.15.110.195:40017)
0927 18:44:12.744138 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.744140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.744149 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:12.744190 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.744191 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.744217 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.744279 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:12.744412 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.744889 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.745581 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.745767 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.745816 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.745952 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:12.746048 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:12.746105 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.746139 (+ 34us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.746155 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:44:12.748039 23602 raft_consensus.cc:1261] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Refusing update from remote peer 7c40d416a7ba44c0b0f231a70eaea64d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:12.748039 23462 raft_consensus.cc:1261] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Refusing update from remote peer 7c40d416a7ba44c0b0f231a70eaea64d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:12.748397 23826 consensus_queue.cc:1035] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:12.748538 23821 consensus_queue.cc:1035] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [LEADER]: Connected to new peer: Peer: permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:12.750723 23829 mvcc.cc:205] Tried to move back new op lower bound from 7334029937653067776 to 7334029937370632192. Current Snapshot: MvccSnapshot[applied={T|T < 7334029937653067776}]
I20260927 18:44:12.751240 23831 mvcc.cc:205] Tried to move back new op lower bound from 7334029937653067776 to 7334029937370632192. Current Snapshot: MvccSnapshot[applied={T|T < 7334029937653067776}]
I20260927 18:44:12.752833 23303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:12.752930 23303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:12.753705 23303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Sending DeleteTablet for 3 replicas of tablet 2f4efbc6769c4944a2d1dc647a140dcb
I20260927 18:44:12.754009 23442 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:12 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36660
I20260927 18:44:12.754083 23582 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:12 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:59852
I20260927 18:44:12.754119 23722 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f4efbc6769c4944a2d1dc647a140dcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:12 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41444
I20260927 18:44:12.754249 23840 tablet_replica.cc:330] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: stopping tablet replica
I20260927 18:44:12.754340 23840 raft_consensus.cc:2229] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:12.754395 23841 tablet_replica.cc:330] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: stopping tablet replica
I20260927 18:44:12.754434 23840 raft_consensus.cc:2258] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:12.754487 23841 raft_consensus.cc:2229] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:12.754580 23841 raft_consensus.cc:2258] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:12.754706 23839 tablet_replica.cc:330] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: stopping tablet replica
I20260927 18:44:12.754794 23839 raft_consensus.cc:2229] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:12.754756 23303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260927 18:44:12.754874 23839 raft_consensus.cc:2258] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 18:44:12.754881 23303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:12.755023 23841 ts_tablet_manager.cc:1905] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:12.755586 23839 ts_tablet_manager.cc:1905] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:12.756063 23442 tablet_service.cc:1467] Processing CreateTablet for tablet d3fad0b4885d4e8b823a3981dd937245 (DEFAULT_TABLE table=test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.756160 23442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fad0b4885d4e8b823a3981dd937245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.757004 23841 ts_tablet_manager.cc:1918] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:12.757066 23841 log.cc:1196] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/2f4efbc6769c4944a2d1dc647a140dcb
I20260927 18:44:12.757122 23839 ts_tablet_manager.cc:1918] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:12.757194 23839 log.cc:1196] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/2f4efbc6769c4944a2d1dc647a140dcb
I20260927 18:44:12.757323 23841 ts_tablet_manager.cc:1939] T 2f4efbc6769c4944a2d1dc647a140dcb P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting consensus metadata
I20260927 18:44:12.757344 23817 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Bootstrap starting.
I20260927 18:44:12.757453 23839 ts_tablet_manager.cc:1939] T 2f4efbc6769c4944a2d1dc647a140dcb P 3a1cdcf228df4d5088f8e674aab76937: Deleting consensus metadata
I20260927 18:44:12.757730 23817 tablet_bootstrap.cc:654] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.758136 23291 catalog_manager.cc:4841] TS 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193): tablet 2f4efbc6769c4944a2d1dc647a140dcb (table test-table [id=2d6f478e2b714e0e97116bea31e2e2de]) successfully deleted
I20260927 18:44:12.758279 23292 catalog_manager.cc:4841] TS 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017): tablet 2f4efbc6769c4944a2d1dc647a140dcb (table test-table [id=2d6f478e2b714e0e97116bea31e2e2de]) successfully deleted
I20260927 18:44:12.758368 23582 tablet_service.cc:1467] Processing CreateTablet for tablet d3fad0b4885d4e8b823a3981dd937245 (DEFAULT_TABLE table=test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.758371 23722 tablet_service.cc:1467] Processing CreateTablet for tablet d3fad0b4885d4e8b823a3981dd937245 (DEFAULT_TABLE table=test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:12.758472 23722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fad0b4885d4e8b823a3981dd937245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.758479 23582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3fad0b4885d4e8b823a3981dd937245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.759021 23817 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: No bootstrap required, opened a new log
I20260927 18:44:12.759068 23817 ts_tablet_manager.cc:1397] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:44:12.759228 23817 raft_consensus.cc:348] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.759308 23817 raft_consensus.cc:374] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.759330 23817 raft_consensus.cc:729] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.759367 23817 consensus_queue.cc:260] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.759676 23817 ts_tablet_manager.cc:1428] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:12.759774 23816 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Bootstrap starting.
I20260927 18:44:12.759845 23822 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Bootstrap starting.
I20260927 18:44:12.760179 23816 tablet_bootstrap.cc:654] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.760190 23822 tablet_bootstrap.cc:654] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.760850 23840 ts_tablet_manager.cc:1905] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:12.760887 23816 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: No bootstrap required, opened a new log
I20260927 18:44:12.760924 23816 ts_tablet_manager.cc:1397] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:12.761050 23816 raft_consensus.cc:348] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.761121 23816 raft_consensus.cc:374] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.761142 23816 raft_consensus.cc:729] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.761196 23816 consensus_queue.cc:260] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.761281 23816 ts_tablet_manager.cc:1428] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.762331 23840 ts_tablet_manager.cc:1918] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:12.762419 23840 log.cc:1196] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/2f4efbc6769c4944a2d1dc647a140dcb
I20260927 18:44:12.762720 23840 ts_tablet_manager.cc:1939] T 2f4efbc6769c4944a2d1dc647a140dcb P 131b6f87839443cc90313c3a1b7146f2: Deleting consensus metadata
I20260927 18:44:12.763118 23292 catalog_manager.cc:4841] TS 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391): tablet 2f4efbc6769c4944a2d1dc647a140dcb (table test-table [id=2d6f478e2b714e0e97116bea31e2e2de]) successfully deleted
I20260927 18:44:12.763378 23822 tablet_bootstrap.cc:492] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: No bootstrap required, opened a new log
I20260927 18:44:12.763418 23822 ts_tablet_manager.cc:1397] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 18:44:12.763547 23822 raft_consensus.cc:348] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.763602 23822 raft_consensus.cc:374] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.763633 23822 raft_consensus.cc:729] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.763677 23822 consensus_queue.cc:260] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.763775 23822 ts_tablet_manager.cc:1428] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.899348 23820 raft_consensus.cc:482] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:12.899447 23820 raft_consensus.cc:504] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.899860 23820 leader_election.cc:288] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:12.904040 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.900356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:34091 (local address 127.15.110.195:40017)
0927 18:44:12.900414 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.900416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.900483 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:44:12.900532 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.900533 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.900542 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.900613 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.900748 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.901438 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903101 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.903669 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903752 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.903763 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.903928 (+ 165us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.903976 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:44:12.904060 23846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.900279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40017 (local address 127.15.110.193:34091)
0927 18:44:12.900416 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.900426 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.900473 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.900632 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.900632 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.900634 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.900684 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.900685 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.901507 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.901508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.902930 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.902933 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.903783 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.903857 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.903862 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.903921 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.903965 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":69,"thread_start_us":99,"threads_started":1}
I20260927 18:44:12.904165 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.900159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39291 (local address 127.15.110.194:37391)
0927 18:44:12.900216 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.900218 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.900254 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:44:12.900332 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.900334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.900344 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.900419 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:12.900598 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.901387 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903087 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.903726 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903787 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.903794 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.904059 (+ 265us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.904103 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:44:12.904176 23845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.900081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:37391 (local address 127.15.110.193:39291)
0927 18:44:12.900193 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.900206 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.900271 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.900441 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.900442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.900444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:12.900524 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.900526 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.901489 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.901490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.902933 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.902933 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.903845 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.903846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.903956 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.903962 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.904047 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.904091 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":57,"thread_start_us":80,"threads_started":1}
I20260927 18:44:12.904335 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fad0b4885d4e8b823a3981dd937245" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2" is_pre_election: true
I20260927 18:44:12.904417 23602 raft_consensus.cc:2454] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 0.
I20260927 18:44:12.904440 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fad0b4885d4e8b823a3981dd937245" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" is_pre_election: true
I20260927 18:44:12.904608 23742 raft_consensus.cc:2454] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 0.
I20260927 18:44:12.904621 23396 leader_election.cc:302] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 3a1cdcf228df4d5088f8e674aab76937; no voters:
I20260927 18:44:12.904744 23820 raft_consensus.cc:2781] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:12.904783 23820 raft_consensus.cc:482] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:12.904807 23820 raft_consensus.cc:3037] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.905519 23820 raft_consensus.cc:504] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.905653 23820 leader_election.cc:288] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 election: Requested vote from peers 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:12.905884 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fad0b4885d4e8b823a3981dd937245" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
I20260927 18:44:12.905965 23742 raft_consensus.cc:3037] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.906157 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3fad0b4885d4e8b823a3981dd937245" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2"
I20260927 18:44:12.906245 23602 raft_consensus.cc:3037] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.906752 23742 raft_consensus.cc:2454] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 1.
I20260927 18:44:12.906809 23602 raft_consensus.cc:2454] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 1.
I20260927 18:44:12.906934 23396 leader_election.cc:302] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1cdcf228df4d5088f8e674aab76937, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:12.907015 23820 raft_consensus.cc:2781] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:12.907132 23820 raft_consensus.cc:686] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Running, Role: LEADER
I20260927 18:44:12.907238 23820 consensus_queue.cc:237] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.907943 23303 catalog_manager.cc:5495] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193). New cstate: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } health_report { overall_health: UNKNOWN } } }
Sep 27 18:44:12 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534651, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.254@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:12.962741 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.955870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.15.110.254:46681)
0927 18:44:12.956021 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.956023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.956032 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:12.956096 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.956096 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.956106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.956167 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:12.956595 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.957100 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.957766 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.957956 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.958004 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.960146 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:12.960147 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:12.960148 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:12.960154 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:12.961648 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.962019 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.962019 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.962020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.962074 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:12.962244 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:12.962244 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:12.962245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:12.962346 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:12.962396 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.962547 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.962643 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
I20260927 18:44:12.962801 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.955806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46681 (local address 127.0.0.1:56396)
0927 18:44:12.955962 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.955975 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.956043 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.956186 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.956186 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.956488 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:12.956535 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.956537 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.957173 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.957173 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.957700 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.957701 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.958017 (+ 316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.958018 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.958056 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.958544 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:12.958548 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:12.960030 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:12.961728 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.961735 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.961737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.961944 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.962134 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:12.962135 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:12.962135 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:12.962190 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:12.962398 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:12.962404 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:12.962455 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.962506 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.962681 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":42,"thread_start_us":78,"threads_started":1}
I20260927 18:44:12.964797 23303 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:56396:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260927 18:44:12.965047 23303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:12.966572 23442 tablet_service.cc:1467] Processing CreateTablet for tablet 09cefd4b8a7845eb8497aacffaea302b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32a2b43507be49f0b3d3ce34af0e117a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:44:12.966576 23722 tablet_service.cc:1467] Processing CreateTablet for tablet 09cefd4b8a7845eb8497aacffaea302b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32a2b43507be49f0b3d3ce34af0e117a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:44:12.966629 23582 tablet_service.cc:1467] Processing CreateTablet for tablet 09cefd4b8a7845eb8497aacffaea302b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32a2b43507be49f0b3d3ce34af0e117a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260927 18:44:12.966742 23722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cefd4b8a7845eb8497aacffaea302b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.966742 23442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cefd4b8a7845eb8497aacffaea302b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.966786 23582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 09cefd4b8a7845eb8497aacffaea302b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:12.968098 23822 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: Bootstrap starting.
I20260927 18:44:12.968238 23816 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: Bootstrap starting.
I20260927 18:44:12.968536 23816 tablet_bootstrap.cc:654] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.968536 23822 tablet_bootstrap.cc:654] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.968865 23817 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: Bootstrap starting.
I20260927 18:44:12.969031 23816 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: No bootstrap required, opened a new log
I20260927 18:44:12.969105 23816 ts_tablet_manager.cc:1397] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:12.969201 23817 tablet_bootstrap.cc:654] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:12.969228 23816 raft_consensus.cc:348] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.969305 23816 raft_consensus.cc:374] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.969331 23816 raft_consensus.cc:729] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.969378 23816 consensus_queue.cc:260] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.969472 23816 ts_tablet_manager.cc:1428] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.969501 23834 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:12.969611 23834 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:12.970063 23817 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: No bootstrap required, opened a new log
I20260927 18:44:12.970129 23817 ts_tablet_manager.cc:1397] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:12.970294 23817 raft_consensus.cc:348] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.970350 23817 raft_consensus.cc:374] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.970364 23817 raft_consensus.cc:729] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.970420 23817 consensus_queue.cc:260] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.970515 23817 ts_tablet_manager.cc:1428] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:12.970563 23820 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:12.970654 23820 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:12.970763 23822 tablet_bootstrap.cc:492] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: No bootstrap required, opened a new log
I20260927 18:44:12.970816 23822 ts_tablet_manager.cc:1397] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:44:12.970960 23822 raft_consensus.cc:348] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.971025 23822 raft_consensus.cc:374] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:12.971040 23822 raft_consensus.cc:729] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Initialized, Role: FOLLOWER
I20260927 18:44:12.971093 23822 consensus_queue.cc:260] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.971239 23822 ts_tablet_manager.cc:1428] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:12.971297 23824 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:12.971408 23824 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:12.984700 23820 raft_consensus.cc:482] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:12.984809 23820 raft_consensus.cc:504] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.984995 23820 leader_election.cc:288] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:12.985287 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cefd4b8a7845eb8497aacffaea302b" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" is_pre_election: true
I20260927 18:44:12.985265 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cefd4b8a7845eb8497aacffaea302b" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2" is_pre_election: true
I20260927 18:44:12.985385 23602 raft_consensus.cc:2454] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 0.
I20260927 18:44:12.985385 23742 raft_consensus.cc:2454] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 0.
I20260927 18:44:12.985527 23396 leader_election.cc:302] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 3a1cdcf228df4d5088f8e674aab76937; no voters:
I20260927 18:44:12.985662 23820 raft_consensus.cc:2781] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:12.985709 23820 raft_consensus.cc:482] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:12.985724 23820 raft_consensus.cc:3037] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.986357 23820 raft_consensus.cc:504] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.986510 23820 leader_election.cc:288] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 election: Requested vote from peers 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:12.986680 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cefd4b8a7845eb8497aacffaea302b" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
I20260927 18:44:12.986757 23742 raft_consensus.cc:3037] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.986707 23602 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "09cefd4b8a7845eb8497aacffaea302b" candidate_uuid: "3a1cdcf228df4d5088f8e674aab76937" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "131b6f87839443cc90313c3a1b7146f2"
I20260927 18:44:12.986822 23602 raft_consensus.cc:3037] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:12.987447 23742 raft_consensus.cc:2454] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 1.
I20260927 18:44:12.987573 23602 raft_consensus.cc:2454] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3a1cdcf228df4d5088f8e674aab76937 in term 1.
I20260927 18:44:12.987617 23396 leader_election.cc:302] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3a1cdcf228df4d5088f8e674aab76937, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:12.987720 23820 raft_consensus.cc:2781] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:12.987785 23820 raft_consensus.cc:686] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [term 1 LEADER]: Becoming Leader. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Running, Role: LEADER
I20260927 18:44:12.987831 23820 consensus_queue.cc:237] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:12.988165 23847 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: TxnStatusTablet state changed. Reason: New leader 3a1cdcf228df4d5088f8e674aab76937. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:12.988255 23847 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:44:12.988451 23305 catalog_manager.cc:5495] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193). New cstate: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } health_report { overall_health: UNKNOWN } } }
I20260927 18:44:12.988479 23860 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260927 18:44:12.988631 23860 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260927 18:44:12.999341 23854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.996720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.0.0.1:36670)
0927 18:44:12.996798 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:12.996813 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:12.996863 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:12.997080 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:12.997081 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:12.997082 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:44:12.997134 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.997136 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.997753 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.997754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.998396 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:12.998398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.998749 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.998750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:12.998806 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.998822 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:44:12.999124 (+ 302us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:44:12.999135 (+ 11us) client_negotiation.cc:769] Sending connection context
0927 18:44:12.999217 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:44:12.999268 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 18:44:12.999325 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:12.996786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36670 (local address 127.15.110.193:36193)
0927 18:44:12.996847 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:12.996849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:12.996858 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:12.996945 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:12.996946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:12.996972 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:12.997072 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:12.997202 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.997670 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.998484 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:12.998675 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:12.998751 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:12.998940 (+ 189us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:12.999053 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:12.999139 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:12.999222 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:44:12.999244 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:44:13.000383 23602 raft_consensus.cc:1261] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Refusing update from remote peer 3a1cdcf228df4d5088f8e674aab76937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.000403 23742 raft_consensus.cc:1261] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Refusing update from remote peer 3a1cdcf228df4d5088f8e674aab76937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.000720 23820 consensus_queue.cc:1035] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.000878 23847 consensus_queue.cc:1035] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.001506 23834 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: TxnStatusTablet state changed. Reason: New leader 3a1cdcf228df4d5088f8e674aab76937. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.001600 23834 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:13.001595 23824 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: TxnStatusTablet state changed. Reason: New leader 3a1cdcf228df4d5088f8e674aab76937. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.001686 23824 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:13.002429 23820 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.002534 23820 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:44:13.002826 23830 mvcc.cc:205] Tried to move back new op lower bound from 7334029938687459328 to 7334029938638430208. Current Snapshot: MvccSnapshot[applied={T|T < 7334029938687459328}]
I20260927 18:44:13.002931 23824 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.002995 23824 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 131b6f87839443cc90313c3a1b7146f2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:13.003472 23847 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.003557 23847 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 3a1cdcf228df4d5088f8e674aab76937: This TxnStatusTablet replica's current role is: LEADER
I20260927 18:44:13.003739 23834 tablet_replica.cc:439] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "3a1cdcf228df4d5088f8e674aab76937" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } } }
I20260927 18:44:13.003822 23834 tablet_replica.cc:442] T 09cefd4b8a7845eb8497aacffaea302b P 7c40d416a7ba44c0b0f231a70eaea64d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260927 18:44:13.007624 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.004323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36682 (local address 127.15.110.193:36193)
0927 18:44:13.004361 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.004363 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.004388 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:44:13.004466 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.004466 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.004476 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.004533 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:13.004650 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.005406 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.006594 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.006997 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.007060 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.007293 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.007331 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.007375 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.007451 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.007514 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:44:13.014927 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.011467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39647 (local address 127.15.110.193:36193)
0927 18:44:13.011512 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.011514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.011523 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:13.011544 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.011545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.011555 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.011613 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:13.011910 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.012685 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.014044 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.014388 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.014475 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.014657 (+ 182us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.014687 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.014734 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.014789 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.014827 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":20}
I20260927 18:44:13.015213 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.010477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.15.110.193:39647)
0927 18:44:13.010706 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.010719 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:13.010796 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:13.011645 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:13.011647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:13.011665 (+ 18us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:44:13.011749 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.011751 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.012782 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.012783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.013901 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.013904 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.014476 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.014476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.014535 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.014555 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.014732 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.014734 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:13.014785 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:44:13.014843 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260927 18:44:13.018342 23742 raft_consensus.cc:1261] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Refusing update from remote peer 3a1cdcf228df4d5088f8e674aab76937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.018374 23602 raft_consensus.cc:1261] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Refusing update from remote peer 3a1cdcf228df4d5088f8e674aab76937: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.018569 23865 consensus_queue.cc:1035] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.018694 23820 consensus_queue.cc:1035] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [LEADER]: Connected to new peer: Peer: permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.044549 23305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.044670 23305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56386:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.045298 23305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Sending DeleteTablet for 3 replicas of tablet d3fad0b4885d4e8b823a3981dd937245
I20260927 18:44:13.045635 23582 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fad0b4885d4e8b823a3981dd937245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:59852
I20260927 18:44:13.045667 23441 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fad0b4885d4e8b823a3981dd937245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36660
I20260927 18:44:13.045652 23722 tablet_service.cc:1514] Processing DeleteTablet for tablet d3fad0b4885d4e8b823a3981dd937245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41444
I20260927 18:44:13.045739 23840 tablet_replica.cc:330] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: stopping tablet replica
I20260927 18:44:13.045760 23839 tablet_replica.cc:330] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: stopping tablet replica
I20260927 18:44:13.045789 23840 raft_consensus.cc:2229] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.045806 23839 raft_consensus.cc:2229] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:13.045825 23840 raft_consensus.cc:2258] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.045897 23839 raft_consensus.cc:2258] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.045977 23841 tablet_replica.cc:330] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: stopping tablet replica
I20260927 18:44:13.046034 23841 raft_consensus.cc:2229] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.046078 23841 raft_consensus.cc:2258] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.046356 23839 ts_tablet_manager.cc:1905] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.048527 23840 ts_tablet_manager.cc:1905] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.049577 23841 ts_tablet_manager.cc:1905] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.050467 23840 ts_tablet_manager.cc:1918] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.050549 23840 log.cc:1196] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/d3fad0b4885d4e8b823a3981dd937245
I20260927 18:44:13.050799 23840 ts_tablet_manager.cc:1939] T d3fad0b4885d4e8b823a3981dd937245 P 131b6f87839443cc90313c3a1b7146f2: Deleting consensus metadata
I20260927 18:44:13.051136 23841 ts_tablet_manager.cc:1918] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.051221 23841 log.cc:1196] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/d3fad0b4885d4e8b823a3981dd937245
I20260927 18:44:13.051213 23292 catalog_manager.cc:4841] TS 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391): tablet d3fad0b4885d4e8b823a3981dd937245 (table test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]) successfully deleted
I20260927 18:44:13.051362 23839 ts_tablet_manager.cc:1918] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.051429 23839 log.cc:1196] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/d3fad0b4885d4e8b823a3981dd937245
I20260927 18:44:13.051437 23841 ts_tablet_manager.cc:1939] T d3fad0b4885d4e8b823a3981dd937245 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting consensus metadata
I20260927 18:44:13.051705 23839 ts_tablet_manager.cc:1939] T d3fad0b4885d4e8b823a3981dd937245 P 3a1cdcf228df4d5088f8e674aab76937: Deleting consensus metadata
I20260927 18:44:13.051754 23292 catalog_manager.cc:4841] TS 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017): tablet d3fad0b4885d4e8b823a3981dd937245 (table test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]) successfully deleted
I20260927 18:44:13.052039 23291 catalog_manager.cc:4841] TS 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193): tablet d3fad0b4885d4e8b823a3981dd937245 (table test-table [id=4a9aeed7035f497e85c72875e1d2fa9e]) successfully deleted
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23260](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:44:13.072697 23855 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:13.066744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56410 (local address 127.15.110.254:46681)
0927 18:44:13.066791 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.066796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.066939 (+ 143us) server_negotiation.cc:407] Connection header received
0927 18:44:13.066983 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.066984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.067000 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.067073 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:13.067845 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.068634 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.069384 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.069713 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.069790 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.072626 (+ 2836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56410: BlockingRecv error: recv got EOF from 127.0.0.1:56410 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23260](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534653, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM
I20260927 18:44:13.081235 23855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.073578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56422 (local address 127.15.110.254:46681)
0927 18:44:13.073621 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.073623 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.073694 (+ 71us) server_negotiation.cc:407] Connection header received
0927 18:44:13.073737 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.073738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.073751 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.073816 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:13.074252 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.074938 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.075709 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.075987 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.076068 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.078463 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:13.078464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:13.078465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:13.078472 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:13.080113 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:13.080443 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:13.080445 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:13.080445 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:13.080526 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:13.080716 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:13.080717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:13.080718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:13.080854 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:13.080992 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.081056 (+ 64us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.081142 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:44:13.082769 23305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:13.082947 23305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:13.084379 23441 tablet_service.cc:1467] Processing CreateTablet for tablet fc0a7b08f4ac45afa4291358a2758974 (DEFAULT_TABLE table=test-table [id=57d3282a47e743069406915a581b4fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.084380 23582 tablet_service.cc:1467] Processing CreateTablet for tablet fc0a7b08f4ac45afa4291358a2758974 (DEFAULT_TABLE table=test-table [id=57d3282a47e743069406915a581b4fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.084408 23722 tablet_service.cc:1467] Processing CreateTablet for tablet fc0a7b08f4ac45afa4291358a2758974 (DEFAULT_TABLE table=test-table [id=57d3282a47e743069406915a581b4fdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.084514 23582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0a7b08f4ac45afa4291358a2758974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.084514 23441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0a7b08f4ac45afa4291358a2758974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.084553 23722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fc0a7b08f4ac45afa4291358a2758974. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.085925 23822 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Bootstrap starting.
I20260927 18:44:13.085961 23817 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Bootstrap starting.
I20260927 18:44:13.085979 23816 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Bootstrap starting.
I20260927 18:44:13.086321 23816 tablet_bootstrap.cc:654] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.086365 23817 tablet_bootstrap.cc:654] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.086364 23822 tablet_bootstrap.cc:654] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.087055 23817 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: No bootstrap required, opened a new log
I20260927 18:44:13.087107 23817 ts_tablet_manager.cc:1397] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:13.087163 23816 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: No bootstrap required, opened a new log
I20260927 18:44:13.087234 23816 ts_tablet_manager.cc:1397] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:13.087293 23817 raft_consensus.cc:348] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.087375 23817 raft_consensus.cc:374] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.087399 23817 raft_consensus.cc:729] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.087353 23816 raft_consensus.cc:348] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.087423 23816 raft_consensus.cc:374] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.087452 23816 raft_consensus.cc:729] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.087441 23817 consensus_queue.cc:260] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.087491 23816 consensus_queue.cc:260] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.087543 23817 ts_tablet_manager.cc:1428] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:13.087585 23816 ts_tablet_manager.cc:1428] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:13.088337 23822 tablet_bootstrap.cc:492] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: No bootstrap required, opened a new log
I20260927 18:44:13.088382 23822 ts_tablet_manager.cc:1397] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:44:13.088537 23822 raft_consensus.cc:348] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.088603 23822 raft_consensus.cc:374] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.088616 23822 raft_consensus.cc:729] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.088668 23822 consensus_queue.cc:260] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.088761 23822 ts_tablet_manager.cc:1428] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:13.155094 23824 raft_consensus.cc:482] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:13.155241 23824 raft_consensus.cc:504] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.155714 23824 leader_election.cc:288] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:13.159808 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.155906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40017 (local address 127.15.110.194:34075)
0927 18:44:13.156067 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.156078 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:13.156172 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:13.156347 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:13.156348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:13.156350 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:13.156451 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.156453 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.157373 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.157374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.158686 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.158689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.159547 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.159548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.159624 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.159628 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:44:13.159680 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:44:13.159719 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":92,"threads_started":1}
I20260927 18:44:13.159840 23815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.156014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:34075 (local address 127.15.110.195:40017)
0927 18:44:13.156080 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.156082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.156139 (+ 57us) server_negotiation.cc:407] Connection header received
0927 18:44:13.156262 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.156263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.156274 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.156335 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:13.156530 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.157284 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.158797 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.159396 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.159446 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.159455 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.159709 (+ 254us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.159767 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:44:13.160415 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0a7b08f4ac45afa4291358a2758974" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" is_pre_election: true
I20260927 18:44:13.160521 23742 raft_consensus.cc:2454] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 0.
I20260927 18:44:13.160710 23536 leader_election.cc:302] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:13.160858 23824 raft_consensus.cc:2781] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:13.160889 23824 raft_consensus.cc:482] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:13.160916 23824 raft_consensus.cc:3037] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.161644 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.156070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:33347 (local address 127.15.110.193:36193)
0927 18:44:13.156115 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.156117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.156219 (+ 102us) server_negotiation.cc:407] Connection header received
0927 18:44:13.156300 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.156301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.156311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.156360 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:13.156577 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.157277 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.160702 (+ 3425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.161270 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.161348 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.161357 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.161537 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.161577 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:44:13.161775 23824 raft_consensus.cc:504] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.161664 23888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.155907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.15.110.194:33347)
0927 18:44:13.156167 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.156176 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:13.156240 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:13.156406 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:13.156407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:13.156410 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:13.156495 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.156508 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.158704 (+ 2196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.158705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.160389 (+ 1684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.160392 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.161360 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.161361 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.161434 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.161440 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:44:13.161517 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:44:13.161554 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"mutex_wait_us":16,"thread_start_us":92,"threads_started":1}
I20260927 18:44:13.161911 23824 leader_election.cc:288] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 election: Requested vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:13.162094 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0a7b08f4ac45afa4291358a2758974" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
I20260927 18:44:13.162110 23462 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0a7b08f4ac45afa4291358a2758974" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937" is_pre_election: true
I20260927 18:44:13.162164 23742 raft_consensus.cc:3037] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.162178 23462 raft_consensus.cc:2454] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 0.
I20260927 18:44:13.162149 23461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fc0a7b08f4ac45afa4291358a2758974" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937"
I20260927 18:44:13.162253 23461 raft_consensus.cc:3037] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.162878 23742 raft_consensus.cc:2454] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 1.
I20260927 18:44:13.163033 23536 leader_election.cc:302] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:13.163074 23461 raft_consensus.cc:2454] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 1.
I20260927 18:44:13.163128 23824 raft_consensus.cc:2781] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:13.163305 23824 raft_consensus.cc:686] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 1 LEADER]: Becoming Leader. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Running, Role: LEADER
I20260927 18:44:13.163399 23824 consensus_queue.cc:237] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.164196 23303 catalog_manager.cc:5495] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "131b6f87839443cc90313c3a1b7146f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } health_report { overall_health: UNKNOWN } } }
I20260927 18:44:13.181648 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.179140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59854 (local address 127.15.110.194:37391)
0927 18:44:13.179237 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.179239 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.179248 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:13.179290 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.179291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.179320 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.179404 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:13.179603 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.180051 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.180833 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.181024 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.181112 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.181296 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.181409 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.181495 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.181533 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.181560 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:44:13.182771 23742 raft_consensus.cc:1261] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Refusing update from remote peer 131b6f87839443cc90313c3a1b7146f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.182765 23461 raft_consensus.cc:1261] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Refusing update from remote peer 131b6f87839443cc90313c3a1b7146f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.183075 23824 consensus_queue.cc:1035] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.183269 23889 consensus_queue.cc:1035] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.186836 23305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.186923 23305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.187739 23305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Sending DeleteTablet for 3 replicas of tablet fc0a7b08f4ac45afa4291358a2758974
I20260927 18:44:13.188043 23722 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0a7b08f4ac45afa4291358a2758974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41444
I20260927 18:44:13.188135 23841 tablet_replica.cc:330] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: stopping tablet replica
I20260927 18:44:13.188184 23841 raft_consensus.cc:2229] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.188215 23841 pending_rounds.cc:62] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Trying to abort 2 pending ops.
I20260927 18:44:13.188232 23841 pending_rounds.cc:68] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Aborting op as it isn't in flight: id { term: 1 index: 2 } timestamp: 7334029939434307584 op_type: WRITE_OP write_request { tablet_id: "fc0a7b08f4ac45afa4291358a2758974" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false immutable: false } columns { name: "val" type: INT32 is_key: false is_nullable: true immutable: false } } row_operations { rows: "<redacted>""\001\003\00090\000\0001\324\000\000" indirect_data: "<redacted>""" } external_consistency_mode: CLIENT_PROPAGATED authz_token { token_data: "\010\251\312\345\325\006\"8\n\ntest-admin\022*\n 57d3282a47e743069406915a581b4fdd\020\001\030\001 \001(\001" signature: "<redacted>""\002Z\\\222\013r\260\324\256\023\241\010\345\300\035*\315m\253$\037\034\373\370\3274>\337\177\351vQ\327\224\353\233\016sg\020\352\n\271\002K\305\027\032\212\265V:\250l\213\362\261b\233\367\227Q\254\003" signing_key_seq_num: 0 } } request_id { client_id: "6093959ed88e4d5bab639682fc730a3a" seq_no: 0 first_incomplete_seq_no: 0 attempt_no: 0 }
I20260927 18:44:13.188402 23841 pending_rounds.cc:68] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Aborting op as it isn't in flight: id { term: 1 index: 1 } timestamp: 7334029939357913088 op_type: NO_OP noop_request { }
I20260927 18:44:13.188437 23841 raft_consensus.cc:2866] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted
I20260927 18:44:13.188464 23841 raft_consensus.cc:2258] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.188850 23441 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0a7b08f4ac45afa4291358a2758974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36660
I20260927 18:44:13.188941 23839 tablet_replica.cc:330] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: stopping tablet replica
I20260927 18:44:13.189002 23839 raft_consensus.cc:2229] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.189050 23839 raft_consensus.cc:2258] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.189383 23582 tablet_service.cc:1514] Processing DeleteTablet for tablet fc0a7b08f4ac45afa4291358a2758974 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:59852
I20260927 18:44:13.189467 23840 tablet_replica.cc:330] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: stopping tablet replica
I20260927 18:44:13.189514 23840 raft_consensus.cc:2229] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:13.189670 23840 raft_consensus.cc:2258] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.190376 23841 ts_tablet_manager.cc:1905] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.189994 23305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:13.191429 23305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:13.192265 23839 ts_tablet_manager.cc:1905] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.192615 23840 ts_tablet_manager.cc:1905] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.193706 23839 ts_tablet_manager.cc:1918] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:13.193756 23839 log.cc:1196] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/fc0a7b08f4ac45afa4291358a2758974
I20260927 18:44:13.193967 23839 ts_tablet_manager.cc:1939] T fc0a7b08f4ac45afa4291358a2758974 P 3a1cdcf228df4d5088f8e674aab76937: Deleting consensus metadata
I20260927 18:44:13.194056 23840 ts_tablet_manager.cc:1918] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:13.194103 23840 log.cc:1196] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/fc0a7b08f4ac45afa4291358a2758974
I20260927 18:44:13.194332 23840 ts_tablet_manager.cc:1939] T fc0a7b08f4ac45afa4291358a2758974 P 131b6f87839443cc90313c3a1b7146f2: Deleting consensus metadata
I20260927 18:44:13.194569 23291 catalog_manager.cc:4841] TS 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193): tablet fc0a7b08f4ac45afa4291358a2758974 (table test-table [id=57d3282a47e743069406915a581b4fdd]) successfully deleted
I20260927 18:44:13.195591 23441 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0ca351309a49c4b15944279f6806db (DEFAULT_TABLE table=test-table [id=716a79aabcae4de0be48d678d096e672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.195602 23722 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0ca351309a49c4b15944279f6806db (DEFAULT_TABLE table=test-table [id=716a79aabcae4de0be48d678d096e672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.195699 23441 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0ca351309a49c4b15944279f6806db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.196887 23817 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Bootstrap starting.
I20260927 18:44:13.197044 23582 tablet_service.cc:1467] Processing CreateTablet for tablet 3e0ca351309a49c4b15944279f6806db (DEFAULT_TABLE table=test-table [id=716a79aabcae4de0be48d678d096e672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:13.197137 23582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0ca351309a49c4b15944279f6806db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.197212 23817 tablet_bootstrap.cc:654] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.197762 23817 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: No bootstrap required, opened a new log
I20260927 18:44:13.197806 23817 ts_tablet_manager.cc:1397] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:13.197904 23817 raft_consensus.cc:348] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.197959 23817 raft_consensus.cc:374] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.197976 23817 raft_consensus.cc:729] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3a1cdcf228df4d5088f8e674aab76937, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.198015 23817 consensus_queue.cc:260] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.198091 23817 ts_tablet_manager.cc:1428] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:13.198359 23822 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Bootstrap starting.
I20260927 18:44:13.198765 23822 tablet_bootstrap.cc:654] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.199081 23292 catalog_manager.cc:4841] TS 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391): tablet fc0a7b08f4ac45afa4291358a2758974 (table test-table [id=57d3282a47e743069406915a581b4fdd]) successfully deleted
I20260927 18:44:13.199653 23822 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: No bootstrap required, opened a new log
I20260927 18:44:13.199705 23822 ts_tablet_manager.cc:1397] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:13.199836 23822 raft_consensus.cc:348] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.199905 23822 raft_consensus.cc:374] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.199926 23822 raft_consensus.cc:729] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.199971 23822 consensus_queue.cc:260] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.200073 23822 ts_tablet_manager.cc:1428] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:13.200538 23841 ts_tablet_manager.cc:1918] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:13.200618 23841 log.cc:1196] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/fc0a7b08f4ac45afa4291358a2758974
I20260927 18:44:13.200837 23841 ts_tablet_manager.cc:1939] T fc0a7b08f4ac45afa4291358a2758974 P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting consensus metadata
I20260927 18:44:13.200999 23722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e0ca351309a49c4b15944279f6806db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:13.202239 23816 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Bootstrap starting.
I20260927 18:44:13.202471 23292 catalog_manager.cc:4841] TS 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017): tablet fc0a7b08f4ac45afa4291358a2758974 (table test-table [id=57d3282a47e743069406915a581b4fdd]) successfully deleted
I20260927 18:44:13.202605 23816 tablet_bootstrap.cc:654] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:13.203217 23816 tablet_bootstrap.cc:492] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: No bootstrap required, opened a new log
I20260927 18:44:13.203266 23816 ts_tablet_manager.cc:1397] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:13.203372 23816 raft_consensus.cc:348] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.203437 23816 raft_consensus.cc:374] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:13.203459 23816 raft_consensus.cc:729] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c40d416a7ba44c0b0f231a70eaea64d, State: Initialized, Role: FOLLOWER
I20260927 18:44:13.203516 23816 consensus_queue.cc:260] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.203608 23816 ts_tablet_manager.cc:1428] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:13.314671 23889 raft_consensus.cc:482] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:13.314779 23889 raft_consensus.cc:504] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.315052 23889 leader_election.cc:288] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:13.315192 23461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0ca351309a49c4b15944279f6806db" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937" is_pre_election: true
I20260927 18:44:13.315353 23461 raft_consensus.cc:2454] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 0.
I20260927 18:44:13.315340 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0ca351309a49c4b15944279f6806db" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" is_pre_election: true
I20260927 18:44:13.315430 23742 raft_consensus.cc:2454] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 0.
I20260927 18:44:13.315528 23535 leader_election.cc:302] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 3a1cdcf228df4d5088f8e674aab76937; no voters:
I20260927 18:44:13.315670 23889 raft_consensus.cc:2781] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:13.315721 23889 raft_consensus.cc:482] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:13.315742 23889 raft_consensus.cc:3037] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.316532 23889 raft_consensus.cc:504] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.316686 23889 leader_election.cc:288] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 election: Requested vote from peers 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193), 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017)
I20260927 18:44:13.316844 23461 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0ca351309a49c4b15944279f6806db" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3a1cdcf228df4d5088f8e674aab76937"
I20260927 18:44:13.316923 23461 raft_consensus.cc:3037] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.316887 23742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e0ca351309a49c4b15944279f6806db" candidate_uuid: "131b6f87839443cc90313c3a1b7146f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c40d416a7ba44c0b0f231a70eaea64d"
I20260927 18:44:13.316962 23742 raft_consensus.cc:3037] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:13.317755 23742 raft_consensus.cc:2454] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 1.
I20260927 18:44:13.317755 23461 raft_consensus.cc:2454] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 131b6f87839443cc90313c3a1b7146f2 in term 1.
I20260927 18:44:13.317947 23536 leader_election.cc:302] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 131b6f87839443cc90313c3a1b7146f2, 7c40d416a7ba44c0b0f231a70eaea64d; no voters:
I20260927 18:44:13.318073 23889 raft_consensus.cc:2781] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:13.318166 23889 raft_consensus.cc:686] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 1 LEADER]: Becoming Leader. State: Replica: 131b6f87839443cc90313c3a1b7146f2, State: Running, Role: LEADER
I20260927 18:44:13.318223 23889 consensus_queue.cc:237] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } }
I20260927 18:44:13.318897 23305 catalog_manager.cc:5495] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "131b6f87839443cc90313c3a1b7146f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "131b6f87839443cc90313c3a1b7146f2" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 37391 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 } health_report { overall_health: UNKNOWN } } }
I20260927 18:44:13.354854 23812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.352278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:53823 (local address 127.15.110.193:36193)
0927 18:44:13.352332 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.352335 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.352458 (+ 123us) server_negotiation.cc:407] Connection header received
0927 18:44:13.352552 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.352553 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.352563 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.352613 (+ 50us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:13.352719 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.353211 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.354071 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.354304 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.354383 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.354583 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.354639 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.354702 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.354756 (+ 54us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.354778 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:44:13.354898 23901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.352208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.15.110.194:53823)
0927 18:44:13.352402 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.352413 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:13.352474 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:13.352615 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:13.352616 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:13.352618 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:44:13.352670 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.352673 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.353297 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.353298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.353971 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.353973 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.354383 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.354384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.354461 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.354498 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.354706 (+ 208us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.354708 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:13.354754 (+ 46us) client_negotiation.cc:240] Negotiation successful
0927 18:44:13.354808 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":81,"threads_started":1}
I20260927 18:44:13.356928 23461 raft_consensus.cc:1261] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Refusing update from remote peer 131b6f87839443cc90313c3a1b7146f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.356957 23742 raft_consensus.cc:1261] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Refusing update from remote peer 131b6f87839443cc90313c3a1b7146f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:13.357160 23889 consensus_queue.cc:1035] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3a1cdcf228df4d5088f8e674aab76937" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.357245 23824 consensus_queue.cc:1035] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c40d416a7ba44c0b0f231a70eaea64d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40017 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:13.363847 23814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.361716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56703 (local address 127.15.110.194:37391)
0927 18:44:13.361752 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:13.361754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:13.361760 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:44:13.361777 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:13.361778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:13.361786 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:13.361826 (+ 40us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:13.361914 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.362373 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.363134 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.363438 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.363501 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.363641 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.363676 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.363722 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:13.363777 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:44:13.363794 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:44:13.363864 23868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.361513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:37391 (local address 127.15.110.193:56703)
0927 18:44:13.361574 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.361585 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:13.361621 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:13.361837 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:13.361838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:13.361839 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0927 18:44:13.361875 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.361876 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.362461 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.362462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.363016 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:13.363018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:13.363508 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:13.363509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:13.363573 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:13.363580 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0927 18:44:13.363740 (+ 160us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0927 18:44:13.363742 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:13.363776 (+ 34us) client_negotiation.cc:240] Negotiation successful
0927 18:44:13.363812 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 18:44:13.369369 23304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.369462 23304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56422:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:13.370097 23304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05dc5caaca544024b1cef7319ed43170: Sending DeleteTablet for 3 replicas of tablet 3e0ca351309a49c4b15944279f6806db
I20260927 18:44:13.370347 23582 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0ca351309a49c4b15944279f6806db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:59852
I20260927 18:44:13.370389 23722 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0ca351309a49c4b15944279f6806db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41444
I20260927 18:44:13.370393 23441 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e0ca351309a49c4b15944279f6806db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:13 UTC) from {username='oryx', principal='oryx/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36660
I20260927 18:44:13.370437 23840 tablet_replica.cc:330] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: stopping tablet replica
I20260927 18:44:13.370477 23841 tablet_replica.cc:330] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: stopping tablet replica
I20260927 18:44:13.370481 23839 tablet_replica.cc:330] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: stopping tablet replica
I20260927 18:44:13.370484 23840 raft_consensus.cc:2229] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:13.370522 23841 raft_consensus.cc:2229] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.370532 23839 raft_consensus.cc:2229] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:13.370550 23840 raft_consensus.cc:2258] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.370616 23841 raft_consensus.cc:2258] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.370620 23839 raft_consensus.cc:2258] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:13.370884 23840 ts_tablet_manager.cc:1905] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.371057 23841 ts_tablet_manager.cc:1905] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.371384 23839 ts_tablet_manager.cc:1905] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:13.372428 23840 ts_tablet_manager.cc:1918] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.372514 23840 log.cc:1196] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/3e0ca351309a49c4b15944279f6806db
I20260927 18:44:13.372668 23841 ts_tablet_manager.cc:1918] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.372732 23841 log.cc:1196] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/3e0ca351309a49c4b15944279f6806db
I20260927 18:44:13.372762 23840 ts_tablet_manager.cc:1939] T 3e0ca351309a49c4b15944279f6806db P 131b6f87839443cc90313c3a1b7146f2: Deleting consensus metadata
I20260927 18:44:13.372975 23841 ts_tablet_manager.cc:1939] T 3e0ca351309a49c4b15944279f6806db P 7c40d416a7ba44c0b0f231a70eaea64d: Deleting consensus metadata
I20260927 18:44:13.372982 23839 ts_tablet_manager.cc:1918] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260927 18:44:13.373029 23839 log.cc:1196] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/3e0ca351309a49c4b15944279f6806db
I20260927 18:44:13.373092 23292 catalog_manager.cc:4841] TS 131b6f87839443cc90313c3a1b7146f2 (127.15.110.194:37391): tablet 3e0ca351309a49c4b15944279f6806db (table test-table [id=716a79aabcae4de0be48d678d096e672]) successfully deleted
I20260927 18:44:13.373261 23839 ts_tablet_manager.cc:1939] T 3e0ca351309a49c4b15944279f6806db P 3a1cdcf228df4d5088f8e674aab76937: Deleting consensus metadata
I20260927 18:44:13.373258 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23377
I20260927 18:44:13.374788 23292 catalog_manager.cc:4841] TS 7c40d416a7ba44c0b0f231a70eaea64d (127.15.110.195:40017): tablet 3e0ca351309a49c4b15944279f6806db (table test-table [id=716a79aabcae4de0be48d678d096e672]) successfully deleted
W20260927 18:44:13.380115 23291 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:36193 (error 108)
I20260927 18:44:13.380426 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23517
I20260927 18:44:13.381191 23906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:13.380499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:36193 (local address 127.0.0.1:36698)
0927 18:44:13.381086 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:13.381141 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":56,"threads_started":1}
W20260927 18:44:13.382216 23291 catalog_manager.cc:4584] TS 3a1cdcf228df4d5088f8e674aab76937 (127.15.110.193:36193): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e0ca351309a49c4b15944279f6806db: Network error: Client connection negotiation failed: client connection to 127.15.110.193:36193: connect: Connection refused (error 111)
I20260927 18:44:13.386454 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23657
I20260927 18:44:13.392220 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23273
2026-09-27T18:44:13Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2914 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23909](info): set up 2 sockets
Sep 27 18:44:13 dist-test-slave-vs05 krb5kdc[23909](info): commencing operation
krb5kdc: starting...
W20260927 18:44:14.415867 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.003s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:14 dist-test-slave-vs05 krb5kdc[23909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534654, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:44:14.531761 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44347
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:44347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:14.614372 23922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:14.614579 23922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:14.614601 23922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:14.615888 23922 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:14.615943 23922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:14.615958 23922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:14.615990 23922 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:14.616017 23922 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:14.617492 23922 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:44347
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:14.617746 23922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:14.617980 23922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:14.623054 23927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:14.622996 23930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:14.623061 23922 server_base.cc:1034] running on GCE node
W20260927 18:44:14.623028 23928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:14.623567 23922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:14.623868 23922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:14.625039 23922 hybrid_clock.cc:648] HybridClock initialized: now 1790534654625015 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-27T18:44:14Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1236 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:14 dist-test-slave-vs05 krb5kdc[23935](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:14 dist-test-slave-vs05 krb5kdc[23935](info): set up 2 sockets
Sep 27 18:44:14 dist-test-slave-vs05 krb5kdc[23935](info): commencing operation
krb5kdc: starting...
W20260927 18:44:15.668176 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:15 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:44:15.786151 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41719
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:41719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:15.867270 23948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:15.867479 23948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:15.867501 23948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:15.868592 23948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:15.868625 23948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:15.868644 23948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:15.868657 23948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:15.868669 23948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:15.870081 23948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:41719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:15.870275 23948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:15.870524 23948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:15.875396 23953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:15.875428 23954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:15.875427 23956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:15.875623 23948 server_base.cc:1034] running on GCE node
I20260927 18:44:15.875888 23948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:15.876186 23948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:15.877372 23948 hybrid_clock.cc:648] HybridClock initialized: now 1790534655877365 us; error 56 us; skew 500 ppm
Sep 27 18:44:15 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534655, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:15.886019 23948 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:44:15.886554 23948 webserver.cc:460] Webserver started at http://127.15.110.254:35227/ using document root <none> and password file <none>
I20260927 18:44:15.886808 23948 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:15.886879 23948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:15.887037 23948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:15.888222 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "c70cb7d4687d4b7d886c3b6ad51118a8"
format_stamp: "Formatted at 2026-09-27 18:44:15 on dist-test-slave-vs05"
server_key: "42431591da3ecae756d3b5e4c474ffba"
server_key_iv: "505e67ddb4b3968deb3b8628cd1529d2"
server_key_version: "encryptionkey@0"
I20260927 18:44:15.888597 23948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "c70cb7d4687d4b7d886c3b6ad51118a8"
format_stamp: "Formatted at 2026-09-27 18:44:15 on dist-test-slave-vs05"
server_key: "42431591da3ecae756d3b5e4c474ffba"
server_key_iv: "505e67ddb4b3968deb3b8628cd1529d2"
server_key_version: "encryptionkey@0"
I20260927 18:44:15.889967 23948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:15.890834 23963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:15.891062 23948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:15.891177 23948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "c70cb7d4687d4b7d886c3b6ad51118a8"
format_stamp: "Formatted at 2026-09-27 18:44:15 on dist-test-slave-vs05"
server_key: "42431591da3ecae756d3b5e4c474ffba"
server_key_iv: "505e67ddb4b3968deb3b8628cd1529d2"
server_key_version: "encryptionkey@0"
I20260927 18:44:15.891291 23948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:15.917520 23948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:15.919545 23948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:15.919698 23948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:15.923606 23948 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:41719
I20260927 18:44:15.923660 24015 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:41719 every 8 connection(s)
I20260927 18:44:15.924029 23948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:15.924947 24016 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:15.927228 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Bootstrap starting.
I20260927 18:44:15.927762 24016 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:15.927991 24016 log.cc:824] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:15.928642 24016 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: No bootstrap required, opened a new log
I20260927 18:44:15.930137 24016 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:15.930271 24016 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:15.930294 24016 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70cb7d4687d4b7d886c3b6ad51118a8, State: Initialized, Role: FOLLOWER
I20260927 18:44:15.930435 24016 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:15.930521 24016 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:15.930545 24016 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:15.930580 24016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:15.931056 24016 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:15.931209 24016 leader_election.cc:302] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70cb7d4687d4b7d886c3b6ad51118a8; no voters:
I20260927 18:44:15.931339 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 23948
I20260927 18:44:15.931402 24016 leader_election.cc:288] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:15.931458 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:15.931465 24019 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:15.931620 24019 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 LEADER]: Becoming Leader. State: Replica: c70cb7d4687d4b7d886c3b6ad51118a8, State: Running, Role: LEADER
I20260927 18:44:15.931654 24016 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:15.931689 15803 external_mini_cluster.cc:1428] Setting key 68693fbbf014e0cd7cf99fceee5ed590
I20260927 18:44:15.931747 24019 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:15.932039 24019 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70cb7d4687d4b7d886c3b6ad51118a8. Latest consensus state: current_term: 1 leader_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } } }
I20260927 18:44:15.932118 24019 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:15.932324 24021 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } } }
I20260927 18:44:15.932451 24021 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:15.932349 24026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:15.933154 24026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:15.934737 24026 catalog_manager.cc:1347] Generated new cluster ID: 75d5529111d147679da05577bbd757df
I20260927 18:44:15.934792 24026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:44:15 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534655, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:15.945061 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:15.932408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38298 (local address 127.15.110.254:41719)
0927 18:44:15.933843 (+ 1435us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:15.933851 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:15.933876 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:44:15.933962 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:15.933967 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:15.934010 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:15.934161 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:15.934938 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:15.936348 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:15.937378 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:15.938029 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:15.938147 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:15.940986 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:15.940991 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:15.940991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:15.941006 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:15.943090 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:15.943512 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:15.943515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:15.943516 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:15.943647 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:15.943897 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:15.943898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:15.943898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:15.944243 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:15.944379 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:15.944647 (+ 268us) server_negotiation.cc:299] Negotiation successful
0927 18:44:15.944812 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1394,"thread_start_us":79,"threads_started":1}
I20260927 18:44:15.955770 24026 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:15.956521 24026 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:15.964257 24026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Generated new TSK 0
I20260927 18:44:15.964468 24026 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:44:15.992451 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:16.073304 24042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.073489 24042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.073513 24042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.074568 24042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.074608 24042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.074635 24042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:16.076040 24042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.076223 24042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.076406 24042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:16.081413 24047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.081488 24042 server_base.cc:1034] running on GCE node
W20260927 18:44:16.081421 24050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.081409 24048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.081919 24042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.082149 24042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.083287 24042 hybrid_clock.cc:648] HybridClock initialized: now 1790534656083264 us; error 42 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.086118 24042 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:44:16.086535 24042 webserver.cc:460] Webserver started at http://127.15.110.193:44639/ using document root <none> and password file <none>
I20260927 18:44:16.086777 24042 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.086860 24042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.087013 24042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:16.088090 24042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "305998b90e6549649e32cfd16b4ff768"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "320be12e70d28b94e98c8c8d513a71ea"
server_key_iv: "5f47f8b6469801b5d7d5913add3440bf"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.088455 24042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "305998b90e6549649e32cfd16b4ff768"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "320be12e70d28b94e98c8c8d513a71ea"
server_key_iv: "5f47f8b6469801b5d7d5913add3440bf"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.089768 24042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:44:16.090592 24057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.090782 24042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:16.090895 24042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "305998b90e6549649e32cfd16b4ff768"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "320be12e70d28b94e98c8c8d513a71ea"
server_key_iv: "5f47f8b6469801b5d7d5913add3440bf"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.091006 24042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.101405 24042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.103330 24042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.103482 24042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.103711 24042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:16.104023 24042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:16.104077 24042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.104125 24042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:16.104153 24042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.109946 24042 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46045
I20260927 18:44:16.109990 24170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46045 every 8 connection(s)
I20260927 18:44:16.110301 24042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:16.117367 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24042
I20260927 18:44:16.117488 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:16.117779 15803 external_mini_cluster.cc:1428] Setting key 1821cb045af8a1bec3a6a6a77b105bc0
I20260927 18:44:16.120011 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.110987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:33941 (local address 127.15.110.254:41719)
0927 18:44:16.111059 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.111061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.111306 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:44:16.111835 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.111836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.111851 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.111915 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.112847 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.113360 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.114354 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.114581 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.114665 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.116789 (+ 2124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:16.116790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:16.116791 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:16.116798 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:16.118524 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.118942 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.118943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.118943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.119011 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.119327 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.119328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.119329 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.119454 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:16.119532 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:16.119815 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 18:44:16.119923 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:44:16.120180 24173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.110972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:33941)
0927 18:44:16.111159 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.111215 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.111724 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.111990 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.111993 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.112553 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.112746 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.112750 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.113459 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.113460 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.114260 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.114262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.114670 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.114671 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.114799 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.115303 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.115311 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.116698 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:16.118614 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.118622 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.118624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.118854 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.119083 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.119084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.119084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.119260 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.119522 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:16.119530 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:16.119676 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 18:44:16.119777 (+ 101us) client_negotiation.cc:240] Negotiation successful
0927 18:44:16.119938 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":95,"threads_started":1}
I20260927 18:44:16.120671 24171 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41719
I20260927 18:44:16.120786 24171 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:16.120970 24171 heartbeater.cc:511] Master 127.15.110.254:41719 requested a full tablet report, sending...
I20260927 18:44:16.121474 23980 ts_manager.cc:194] Registered new tserver with Master: 305998b90e6549649e32cfd16b4ff768 (127.15.110.193:46045)
I20260927 18:44:16.122102 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:33941
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:44:16.143584 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:16.225243 24176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.225447 24176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.225471 24176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.226563 24176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.226600 24176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.226625 24176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:16.228156 24176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.228348 24176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.228511 24176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:16.233608 24184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.233624 24182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.233628 24181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.233754 24176 server_base.cc:1034] running on GCE node
I20260927 18:44:16.234067 24176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.234313 24176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.235451 24176 hybrid_clock.cc:648] HybridClock initialized: now 1790534656235430 us; error 38 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.237963 24176 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:44:16.238462 24176 webserver.cc:460] Webserver started at http://127.15.110.194:43151/ using document root <none> and password file <none>
I20260927 18:44:16.238706 24176 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.238782 24176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.238941 24176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:16.240046 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "f6a83732017e45ba84b6788acd98a123"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9d0ba35e1ee47ce0c35276c3b6627ccb"
server_key_iv: "46d5742de47f9e0fff01620a7a573312"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.240413 24176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "f6a83732017e45ba84b6788acd98a123"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9d0ba35e1ee47ce0c35276c3b6627ccb"
server_key_iv: "46d5742de47f9e0fff01620a7a573312"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.241772 24176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:44:16.242514 24191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.242738 24176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:16.242834 24176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "f6a83732017e45ba84b6788acd98a123"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9d0ba35e1ee47ce0c35276c3b6627ccb"
server_key_iv: "46d5742de47f9e0fff01620a7a573312"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.242919 24176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.261709 24176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.263757 24176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.263923 24176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.264150 24176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:16.264468 24176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:16.264523 24176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.264566 24176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:16.264593 24176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.270367 24176 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:39563
I20260927 18:44:16.270454 24304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:39563 every 8 connection(s)
I20260927 18:44:16.270750 24176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:16.278429 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24176
I20260927 18:44:16.278509 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:16.278703 15803 external_mini_cluster.cc:1428] Setting key b721897434ce56cae9785ce99c4856e1
I20260927 18:44:16.280385 24307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.271463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:48085)
0927 18:44:16.271658 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.271692 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.272146 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.272355 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.272358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.272913 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.273104 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.273109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.273854 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.273854 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.274636 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.274638 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.275008 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.275008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.275111 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.275668 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.275676 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.277088 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:16.278880 (+ 1792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.278888 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.278891 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.279177 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.279415 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.279416 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.279416 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.279571 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.279888 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:16.279893 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:16.280012 (+ 119us) client_negotiation.cc:769] Sending connection context
0927 18:44:16.280099 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:44:16.280220 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":134,"threads_started":1}
I20260927 18:44:16.280381 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.271480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:48085 (local address 127.15.110.254:41719)
0927 18:44:16.271524 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.271526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.271791 (+ 265us) server_negotiation.cc:407] Connection header received
0927 18:44:16.272241 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.272242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.272251 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.272302 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.273224 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.273758 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.274729 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.274922 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.274988 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.277184 (+ 2196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:16.277186 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:16.277186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:16.277194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:16.278778 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.279264 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.279265 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.279265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.279322 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.279654 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.279656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.279656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.279818 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:16.279888 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:16.280145 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 18:44:16.280243 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:44:16.280992 24305 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41719
I20260927 18:44:16.281107 24305 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:16.281291 24305 heartbeater.cc:511] Master 127.15.110.254:41719 requested a full tablet report, sending...
I20260927 18:44:16.281713 23980 ts_manager.cc:194] Registered new tserver with Master: f6a83732017e45ba84b6788acd98a123 (127.15.110.194:39563)
I20260927 18:44:16.282191 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:48085
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:44:16.307045 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:16.389561 24310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.389797 24310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.389833 24310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.391001 24310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.391067 24310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.391111 24310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:16.392565 24310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.392822 24310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.393049 24310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:16.398097 24316 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.398120 24318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.398272 24315 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.398587 24310 server_base.cc:1034] running on GCE node
I20260927 18:44:16.398824 24310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.399062 24310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.400229 24310 hybrid_clock.cc:648] HybridClock initialized: now 1790534656400212 us; error 28 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.402772 24310 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:44:16.403266 24310 webserver.cc:460] Webserver started at http://127.15.110.195:39311/ using document root <none> and password file <none>
I20260927 18:44:16.403493 24310 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.403558 24310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.403688 24310 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:16.404693 24310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "8f97dd613a1949b5a8e698fe7c242007"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9ea6fc58c6edf4ab083df13756bd6b0a"
server_key_iv: "0c08939d1c87afe8a4e26ec90d9eb4fe"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.405050 24310 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "8f97dd613a1949b5a8e698fe7c242007"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9ea6fc58c6edf4ab083df13756bd6b0a"
server_key_iv: "0c08939d1c87afe8a4e26ec90d9eb4fe"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.406358 24310 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:16.407081 24325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.407269 24310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:16.407373 24310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "8f97dd613a1949b5a8e698fe7c242007"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9ea6fc58c6edf4ab083df13756bd6b0a"
server_key_iv: "0c08939d1c87afe8a4e26ec90d9eb4fe"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.407457 24310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.421766 24310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.424152 24310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.424310 24310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.424541 24310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:16.424836 24310 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:16.424886 24310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.424926 24310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:16.424954 24310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.430830 24310 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:44527
I20260927 18:44:16.430869 24438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:44527 every 8 connection(s)
I20260927 18:44:16.431214 24310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:16.431921 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24310
I20260927 18:44:16.432034 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:16.432294 15803 external_mini_cluster.cc:1428] Setting key b48cd672ecc7de812217db1d7c974120
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:16.440471 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.432012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:38827 (local address 127.15.110.254:41719)
0927 18:44:16.432080 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.432083 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.432328 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:44:16.433061 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.433061 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.433070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.433107 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.434164 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.434656 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.435744 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.435933 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.435989 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.437986 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:16.437988 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:16.437988 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:16.437995 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:16.439278 (+ 1283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.439663 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.439664 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.439664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.439715 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:16.439911 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:16.439912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:16.439912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:16.440011 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:16.440062 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:16.440343 (+ 281us) server_negotiation.cc:299] Negotiation successful
0927 18:44:16.440405 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:44:16.440765 24441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.432020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:38827)
0927 18:44:16.432183 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.432225 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.432969 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.433186 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.433190 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.433888 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.434086 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.434091 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.434746 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.434747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.435669 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.435671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.436007 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.436008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.436099 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.436605 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.436613 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.437908 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:16.439345 (+ 1437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.439350 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.439352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.439601 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.439770 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:16.439771 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:16.439772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:16.439861 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:16.440093 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:16.440100 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:16.440219 (+ 119us) client_negotiation.cc:769] Sending connection context
0927 18:44:16.440319 (+ 100us) client_negotiation.cc:240] Negotiation successful
0927 18:44:16.440439 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260927 18:44:16.441459 24439 heartbeater.cc:347] Connected to a master server at 127.15.110.254:41719
I20260927 18:44:16.441615 24439 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:16.441802 24439 heartbeater.cc:511] Master 127.15.110.254:41719 requested a full tablet report, sending...
I20260927 18:44:16.442195 23980 ts_manager.cc:194] Registered new tserver with Master: 8f97dd613a1949b5a8e698fe7c242007 (127.15.110.195:44527)
I20260927 18:44:16.442615 23980 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:38827
I20260927 18:44:16.444125 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
Entry for principal oryx/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab.
WARNING: no policy specified for oryx/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
Entry for principal oryx/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab.
WARNING: no policy specified for oryx/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
Entry for principal oryx/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab.
I20260927 18:44:16.514264 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24042
I20260927 18:44:16.519083 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24176
I20260927 18:44:16.523283 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24310
I20260927 18:44:16.527110 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 23948
I20260927 18:44:16.531876 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41719
--webserver_interface=127.15.110.254
--webserver_port=35227
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:41719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:16.613858 24448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.614086 24448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.614110 24448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.615453 24448 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:16.615509 24448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.615525 24448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.615536 24448 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:16.615598 24448 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:16.617130 24448 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:41719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=35227
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.617417 24448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.617664 24448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:44:16.623133 24448 server_base.cc:1034] running on GCE node
W20260927 18:44:16.623102 24456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.623095 24453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.623155 24454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.623584 24448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.624174 24448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.625340 24448 hybrid_clock.cc:648] HybridClock initialized: now 1790534656625319 us; error 42 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.628119 24448 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:44:16.628623 24448 webserver.cc:460] Webserver started at http://127.15.110.254:35227/ using document root <none> and password file <none>
I20260927 18:44:16.628896 24448 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.628973 24448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.630343 24448 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:16.631122 24463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.631333 24448 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:16.631443 24448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "c70cb7d4687d4b7d886c3b6ad51118a8"
format_stamp: "Formatted at 2026-09-27 18:44:15 on dist-test-slave-vs05"
server_key: "42431591da3ecae756d3b5e4c474ffba"
server_key_iv: "505e67ddb4b3968deb3b8628cd1529d2"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.631834 24448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.639578 24448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.641697 24448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.641891 24448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.646318 24448 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:41719
I20260927 18:44:16.646350 24515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:41719 every 8 connection(s)
I20260927 18:44:16.646767 24448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:16.648279 24516 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:44:16.648926 24516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Bootstrap starting.
I20260927 18:44:16.656323 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24448
I20260927 18:44:16.656718 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:46045
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:41719
--webserver_port=44639
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:44:16.657238 24516 log.cc:824] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:16.659636 24516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:44:16.659943 24516 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Bootstrap complete.
I20260927 18:44:16.664029 24516 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:16.664450 24516 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70cb7d4687d4b7d886c3b6ad51118a8, State: Initialized, Role: FOLLOWER
I20260927 18:44:16.664646 24516 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:16.664736 24516 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:16.664790 24516 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:16.664852 24516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:44:16.666561 24516 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:16.666697 24516 leader_election.cc:302] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70cb7d4687d4b7d886c3b6ad51118a8; no voters:
I20260927 18:44:16.666887 24516 leader_election.cc:288] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:44:16.667057 24520 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:44:16.667243 24516 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:16.667289 24520 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [term 2 LEADER]: Becoming Leader. State: Replica: c70cb7d4687d4b7d886c3b6ad51118a8, State: Running, Role: LEADER
I20260927 18:44:16.667399 24520 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } }
I20260927 18:44:16.667698 24521 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } } }
I20260927 18:44:16.667784 24521 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:16.668054 24522 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70cb7d4687d4b7d886c3b6ad51118a8. Latest consensus state: current_term: 2 leader_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70cb7d4687d4b7d886c3b6ad51118a8" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41719 } } }
I20260927 18:44:16.668197 24522 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:16.668221 24527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:16.668946 24527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:16.669158 24527 catalog_manager.cc:1259] Loaded cluster ID: 75d5529111d147679da05577bbd757df
I20260927 18:44:16.669296 24527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:16.670593 24527 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:16.670763 24527 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c70cb7d4687d4b7d886c3b6ad51118a8: Loaded TSK: 0
I20260927 18:44:16.670964 24527 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:44:16.743470 24519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.743669 24519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.743692 24519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.744884 24519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.744925 24519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.744952 24519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:16.746377 24519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.193.keytab
--principal=oryx/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:46045
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=44639
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.746610 24519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.746865 24519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:16.752003 24542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.752071 24543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.752058 24545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.752590 24519 server_base.cc:1034] running on GCE node
I20260927 18:44:16.752817 24519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.753074 24519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.754244 24519 hybrid_clock.cc:648] HybridClock initialized: now 1790534656754230 us; error 31 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.756843 24519 init.cc:377] Logged in from keytab as oryx/127.15.110.193@KRBTEST.COM (short username oryx)
I20260927 18:44:16.757339 24519 webserver.cc:460] Webserver started at http://127.15.110.193:44639/ using document root <none> and password file <none>
I20260927 18:44:16.757589 24519 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.757681 24519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.759143 24519 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:16.759912 24552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.760100 24519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:16.760216 24519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "305998b90e6549649e32cfd16b4ff768"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "320be12e70d28b94e98c8c8d513a71ea"
server_key_iv: "5f47f8b6469801b5d7d5913add3440bf"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.760618 24519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.772956 24519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.775646 24519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.775846 24519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.776131 24519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:16.776523 24519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:16.776592 24519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.776638 24519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:16.776669 24519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.784916 24519 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46045
I20260927 18:44:16.784963 24665 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46045 every 8 connection(s)
I20260927 18:44:16.785398 24519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:16.793042 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24519
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.793402 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:39563
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:41719
--webserver_port=43151
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.794932 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.786248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:43707)
0927 18:44:16.786439 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.786481 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.787134 (+ 653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.787672 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.787676 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.788332 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.788539 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.788545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.789788 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.789790 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.790675 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.790677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.791426 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.791427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.791575 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.792192 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.792200 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.794701 (+ 2501us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":92,"threads_started":1}
W20260927 18:44:16.795239 24666 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:41719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.110.254:41719: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
W20260927 18:44:16.795382 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.786551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:43707 (local address 127.15.110.254:41719)
0927 18:44:16.786766 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.786775 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.786820 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:44:16.787411 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.787417 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.787463 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.787632 (+ 169us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.788674 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.789672 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.790793 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.791311 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.791451 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.794828 (+ 3377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:43707: BlockingRecv error: recv got EOF from 127.15.110.193:43707 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":86,"threads_started":1}
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.800724 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.795474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:52297)
0927 18:44:16.795799 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.795810 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.795885 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.796094 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.796095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.796229 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.796274 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.796275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.796967 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.796969 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.797591 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.797594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.798152 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.798153 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.798280 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.798765 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.798771 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.800621 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":79,"threads_started":1}
W20260927 18:44:16.800774 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.795665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:52297 (local address 127.15.110.254:41719)
0927 18:44:16.795728 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.795730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.795869 (+ 139us) server_negotiation.cc:407] Connection header received
0927 18:44:16.795952 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.795953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.795968 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.796062 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.796360 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.796878 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.797707 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.797916 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.798098 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.800705 (+ 2607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:52297: BlockingRecv error: recv got EOF from 127.15.110.193:52297 (error 108)
Metrics: {"server-negotiator.queue_time_us":44,"thread_start_us":77,"threads_started":1}
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.805826 24668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.801299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:54047)
0927 18:44:16.801345 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.801355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.801396 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.801553 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.801554 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.801700 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.801728 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.801729 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.802396 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.802397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.802957 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.802958 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.803423 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.803425 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.803482 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.803992 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.803997 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.805737 (+ 1740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:44:16.805872 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.801360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:54047 (local address 127.15.110.254:41719)
0927 18:44:16.801427 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.801429 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.801437 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:16.801461 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.801462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.801472 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.801550 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.801808 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.802311 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.803048 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.803331 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.803423 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.805808 (+ 2385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:54047: BlockingRecv error: recv got EOF from 127.15.110.193:54047 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260927 18:44:16.806293 24666 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:44:16.885416 24670 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:16.885622 24670 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:16.885645 24670 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:16.886740 24670 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:16.886775 24670 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:16.886799 24670 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:16.888175 24670 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.194.keytab
--principal=oryx/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:39563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=43151
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24670
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:16.888360 24670 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:16.888530 24670 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:44:16.893735 24670 server_base.cc:1034] running on GCE node
W20260927 18:44:16.893707 24680 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.893698 24677 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:16.893705 24678 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:16.894150 24670 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:16.894410 24670 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:16.895594 24670 hybrid_clock.cc:648] HybridClock initialized: now 1790534656895577 us; error 43 us; skew 500 ppm
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534656, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:16.898324 24670 init.cc:377] Logged in from keytab as oryx/127.15.110.194@KRBTEST.COM (short username oryx)
I20260927 18:44:16.898725 24670 webserver.cc:460] Webserver started at http://127.15.110.194:43151/ using document root <none> and password file <none>
I20260927 18:44:16.898910 24670 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:16.898964 24670 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:16.900393 24670 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:16.900985 24687 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.901160 24670 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:16.901252 24670 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "f6a83732017e45ba84b6788acd98a123"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9d0ba35e1ee47ce0c35276c3b6627ccb"
server_key_iv: "46d5742de47f9e0fff01620a7a573312"
server_key_version: "encryptionkey@0"
I20260927 18:44:16.901604 24670 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:16.926226 24670 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:16.928375 24670 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:16.928553 24670 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:16.928797 24670 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:16.929121 24670 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:16.929183 24670 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.929235 24670 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:16.929265 24670 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:16.935535 24670 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:39563
I20260927 18:44:16.935659 24800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:39563 every 8 connection(s)
I20260927 18:44:16.935985 24670 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:16.939742 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24670
I20260927 18:44:16.940038 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:44527
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:41719
--webserver_port=39311
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.944340 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.936855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:47209)
0927 18:44:16.937034 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.937071 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.937553 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.937802 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.937806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.938385 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.938585 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.938589 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.939376 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.939377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.940342 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.940345 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.940828 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.940829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.940930 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.941520 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.941530 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.944189 (+ 2659us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
W20260927 18:44:16.944407 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.936857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:47209 (local address 127.15.110.254:41719)
0927 18:44:16.936912 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.936915 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.937175 (+ 260us) server_negotiation.cc:407] Connection header received
0927 18:44:16.937672 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.937673 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.937689 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.937747 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.938704 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.939278 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.940425 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.940743 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.940819 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.944320 (+ 3501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:47209: BlockingRecv error: recv got EOF from 127.15.110.194:47209 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:44:16.944705 24801 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:41719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.110.254:41719: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.950192 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.944939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:51153)
0927 18:44:16.945004 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.945014 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.945054 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.945201 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.945202 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.945367 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.945426 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.945428 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.946095 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.946096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.947032 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.947034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.947448 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.947450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.947526 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.947961 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.947967 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.950094 (+ 2127us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:44:16.950328 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.945017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:51153 (local address 127.15.110.254:41719)
0927 18:44:16.945069 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.945071 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.945079 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:16.945113 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.945114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.945122 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.945173 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.945555 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.946018 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.947125 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.947374 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.947432 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.950222 (+ 2790us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:51153: BlockingRecv error: recv got EOF from 127.15.110.194:51153 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:16 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:16.955451 24803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:16.950450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:49389)
0927 18:44:16.950581 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:16.950592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:16.950645 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:16.950802 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:16.950803 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:16.950955 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:16.951004 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.951006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.951740 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.951741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.952555 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:16.952557 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.952935 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.952936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:16.952996 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.953396 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:16.953401 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:16.955373 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":77,"threads_started":1}
W20260927 18:44:16.955542 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:16.950511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:49389 (local address 127.15.110.254:41719)
0927 18:44:16.950549 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:16.950551 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:16.950654 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:44:16.950705 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:16.950706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:16.950716 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:16.950791 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:16.951083 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.951660 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.952655 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:16.952858 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:16.952926 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:16.955459 (+ 2533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:49389: BlockingRecv error: recv got EOF from 127.15.110.194:49389 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:44:16.955578 24801 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:44:17.027511 24804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:17.027716 24804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:17.027740 24804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:17.028860 24804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:17.028895 24804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:17.028921 24804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:17.030331 24804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/krb5kdc/oryx_127.15.110.195.keytab
--principal=oryx/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:44527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=39311
--tserver_master_addrs=127.15.110.254:41719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:17.030524 24804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:17.030699 24804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:17.036070 24810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:17.036207 24804 server_base.cc:1034] running on GCE node
W20260927 18:44:17.036129 24811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:17.036091 24813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:17.036710 24804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:17.036991 24804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:17.038161 24804 hybrid_clock.cc:648] HybridClock initialized: now 1790534657038136 us; error 32 us; skew 500 ppm
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534657, etypes {rep=17 tkt=17 ses=17}, oryx/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:17.040824 24804 init.cc:377] Logged in from keytab as oryx/127.15.110.195@KRBTEST.COM (short username oryx)
I20260927 18:44:17.041319 24804 webserver.cc:460] Webserver started at http://127.15.110.195:39311/ using document root <none> and password file <none>
I20260927 18:44:17.041556 24804 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:17.041646 24804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:17.043073 24804 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:17.043813 24820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:17.044003 24804 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:44:17.044107 24804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "8f97dd613a1949b5a8e698fe7c242007"
format_stamp: "Formatted at 2026-09-27 18:44:16 on dist-test-slave-vs05"
server_key: "9ea6fc58c6edf4ab083df13756bd6b0a"
server_key_iv: "0c08939d1c87afe8a4e26ec90d9eb4fe"
server_key_version: "encryptionkey@0"
I20260927 18:44:17.044467 24804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:17.064419 24804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:17.067050 24804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:17.067343 24804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:17.067634 24804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:17.067970 24804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:17.068030 24804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:17.068080 24804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:17.068133 24804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:17.074544 24804 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:44527
I20260927 18:44:17.074596 24933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:44527 every 8 connection(s)
I20260927 18:44:17.074990 24804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:17.076063 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 24804
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
W20260927 18:44:17.083936 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:17.075879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:60209 (local address 127.15.110.254:41719)
0927 18:44:17.075959 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.075962 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.076251 (+ 289us) server_negotiation.cc:407] Connection header received
0927 18:44:17.077064 (+ 813us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.077066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.077079 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.077164 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.078350 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.079224 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.080431 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.080636 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.080720 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.083877 (+ 3157us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:60209: BlockingRecv error: recv got EOF from 127.15.110.195:60209 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
I20260927 18:44:17.084013 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.075899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:60209)
0927 18:44:17.076075 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:17.076114 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:17.076951 (+ 837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:17.077219 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:17.077222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:17.077954 (+ 732us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:17.078226 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.078233 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.079314 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.079315 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.080335 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.080338 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.080733 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.080734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.080856 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.081395 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:17.081404 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:17.083810 (+ 2406us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":95,"threads_started":1}
W20260927 18:44:17.084234 24934 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:41719 (0 consecutive failures): Not authorized: Failed to ping master at 127.15.110.254:41719: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
I20260927 18:44:17.085371 24672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.076600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38300 (local address 127.15.110.254:41719)
0927 18:44:17.076650 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.076652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.076662 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:17.076719 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.076720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.076733 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.076816 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.077583 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.078437 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.079573 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.079897 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.079979 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.081397 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:17.081402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:17.081403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:17.081413 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:17.083559 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:17.083992 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:17.083995 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:17.083996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:17.084148 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:17.084396 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:17.084397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:17.084397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:17.084738 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:17.084842 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:17.085148 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:44:17.085267 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:17.089325 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.084470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:49819)
0927 18:44:17.084524 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:17.084535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:17.084578 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:17.084742 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:17.084743 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:17.084937 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:17.084996 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.084999 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.085855 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.085856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.086499 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.086502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.087044 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.087045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.087106 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.087515 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:17.087519 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:17.089279 (+ 1760us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:44:17.089458 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:17.084524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49819 (local address 127.15.110.254:41719)
0927 18:44:17.084558 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.084560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.084569 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:17.084626 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.084627 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.084635 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.084694 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.085090 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.085781 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.086640 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.086950 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.087008 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.089370 (+ 2362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:49819: BlockingRecv error: recv got EOF from 127.15.110.195:49819 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:17.093878 24936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.089730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:32939)
0927 18:44:17.089805 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:17.089819 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:17.089862 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:17.089989 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:17.089990 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:17.090127 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:17.090153 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.090155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.090811 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.090812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.091430 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.091433 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.091798 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.091799 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.091839 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.092194 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:17.092198 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:17.093816 (+ 1618us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260927 18:44:17.093940 24669 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:17.089802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:32939 (local address 127.15.110.254:41719)
0927 18:44:17.089841 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.089843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.089852 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:17.089924 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.089924 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.089932 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.089982 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.090218 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.090718 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.091539 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.091728 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.091787 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.093899 (+ 2112us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:32939: BlockingRecv error: recv got EOF from 127.15.110.195:32939 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 18:44:17.094182 24934 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:17.811262 24938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.806689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:51737)
0927 18:44:17.806824 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:17.806835 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:17.806885 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:17.807004 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:17.807004 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:17.807168 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:17.807286 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.807288 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.808011 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.808012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.808584 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.808586 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.808968 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.808969 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.809018 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.809449 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:17.809453 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:17.811195 (+ 1742us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
W20260927 18:44:17.811367 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:17.806757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:51737 (local address 127.15.110.254:41719)
0927 18:44:17.806856 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.806858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.806873 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:44:17.806929 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.806929 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.806939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.806992 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.807364 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.807916 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.808687 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.808882 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.808945 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.811295 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:51737: BlockingRecv error: recv got EOF from 127.15.110.193:51737 (error 108)
Metrics: {"server-negotiator.queue_time_us":75,"thread_start_us":48,"threads_started":1}
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:17 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:17.960820 24940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:17.955962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:59541)
0927 18:44:17.956136 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:17.956163 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:17.956249 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:17.956381 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:17.956382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:17.956557 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:17.956604 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.956606 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.957288 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.957289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.957837 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:17.957838 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.958201 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.958202 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:17.958260 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.958806 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:17.958818 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:17.960753 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":71,"threads_started":1}
W20260927 18:44:17.960919 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:17.956016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:59541 (local address 127.15.110.254:41719)
0927 18:44:17.956096 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:17.956098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:17.956228 (+ 130us) server_negotiation.cc:407] Connection header received
0927 18:44:17.956303 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:17.956303 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:17.956317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:17.956371 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:17.956706 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.957205 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.957925 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:17.958127 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:17.958203 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:17.960851 (+ 2648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:59541: BlockingRecv error: recv got EOF from 127.15.110.194:59541 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:18.099044 24941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:18.094588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:46175)
0927 18:44:18.094764 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:18.094774 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:18.094836 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:18.094973 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:18.094974 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:18.095127 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:18.095197 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.095199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.095827 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.095828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.096383 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.096385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.096695 (+ 310us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.096696 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.096732 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.097141 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:18.097145 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:18.098976 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":114,"threads_started":1}
W20260927 18:44:18.099217 24939 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:18.094654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:46175 (local address 127.15.110.254:41719)
0927 18:44:18.094750 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:18.094754 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:18.094821 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:44:18.094890 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:18.094891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:18.094904 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:18.094947 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:18.095263 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.095754 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.096452 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.096632 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.096682 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.099094 (+ 2412us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:46175: BlockingRecv error: recv got EOF from 127.15.110.195:46175 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:18.816349 24942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:18.811761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:43659)
0927 18:44:18.811912 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:18.811923 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:18.811971 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:18.812119 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:18.812119 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:18.812284 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:18.812334 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.812335 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.812962 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.812962 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.813477 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.813479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.813800 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.813801 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.813841 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.814286 (+ 445us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:18.814291 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:18.816289 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":88,"threads_started":1}
W20260927 18:44:18.816471 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:18.811844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:43659 (local address 127.15.110.254:41719)
0927 18:44:18.811986 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:18.811988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:18.812005 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:44:18.812026 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:18.812027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:18.812037 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:18.812097 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:18.812402 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.812895 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.813549 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.813737 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.813786 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.816365 (+ 2579us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:43659: BlockingRecv error: recv got EOF from 127.15.110.193:43659 (error 108)
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":91,"threads_started":1}
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:18 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:18.965862 24944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:18.961369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:45929)
0927 18:44:18.961552 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:18.961563 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:18.961623 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:18.961763 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:18.961763 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:18.961923 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:18.961973 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.961974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.962618 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.962619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.963163 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:18.963165 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.963601 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.963602 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:18.963648 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.964055 (+ 407us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:18.964059 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:18.965800 (+ 1741us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
W20260927 18:44:18.965992 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:18.961419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:45929 (local address 127.15.110.254:41719)
0927 18:44:18.961492 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:18.961494 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:18.961628 (+ 134us) server_negotiation.cc:407] Connection header received
0927 18:44:18.961678 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:18.961679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:18.961690 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:18.961748 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:18.962037 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.962550 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.963296 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:18.963521 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:18.963598 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:18.965910 (+ 2312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:45929: BlockingRecv error: recv got EOF from 127.15.110.194:45929 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:19.104180 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:19.099541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:41707)
0927 18:44:19.099676 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:19.099688 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:19.099744 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:19.099869 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:19.099869 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:19.100034 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:19.100082 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.100083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.100703 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.100704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.101243 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.101245 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.101580 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.101581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.101620 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.102115 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:19.102120 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:19.104081 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":86,"threads_started":1}
W20260927 18:44:19.104234 24943 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:19.099664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:41707 (local address 127.15.110.254:41719)
0927 18:44:19.099721 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:19.099723 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:19.099739 (+ 16us) server_negotiation.cc:407] Connection header received
0927 18:44:19.099789 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:19.099790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:19.099801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:19.099849 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:19.100142 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.100625 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.101326 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.101511 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.101569 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.104170 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:41707: BlockingRecv error: recv got EOF from 127.15.110.195:41707 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:19.821538 24946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:19.816883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:54753)
0927 18:44:19.817038 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:19.817050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:19.817103 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:19.817260 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:19.817261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:19.817427 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:19.817475 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.817477 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.818187 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.818188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.818701 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.818703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.819043 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.819044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.819082 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.819618 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:19.819624 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:19.821479 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":90,"threads_started":1}
W20260927 18:44:19.821774 24947 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:19.816976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:54753 (local address 127.15.110.254:41719)
0927 18:44:19.817077 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:19.817079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:19.817089 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:19.817170 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:19.817171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:19.817181 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:19.817240 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:19.817573 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.818112 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.818784 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.818978 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.819031 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.821705 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:54753: BlockingRecv error: recv got EOF from 127.15.110.193:54753 (error 108)
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:19 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:19.971390 24948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:19.966319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:33709)
0927 18:44:19.966453 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:19.966469 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:19.966518 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:19.966694 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:19.966695 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:19.966855 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:19.966902 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.966903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.967794 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.967795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.968327 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:19.968329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.968775 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.968776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:19.968842 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.969376 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:19.969380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:19.971328 (+ 1948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":81,"threads_started":1}
W20260927 18:44:19.971503 24947 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:19.966454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:33709 (local address 127.15.110.254:41719)
0927 18:44:19.966536 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:19.966539 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:19.966549 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:19.966582 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:19.966583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:19.966596 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:19.966678 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:19.966974 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.967714 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.968429 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:19.968694 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:19.968774 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:19.971428 (+ 2654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:33709: BlockingRecv error: recv got EOF from 127.15.110.194:33709 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:20.109319 24949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:20.104671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:32775)
0927 18:44:20.104851 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:20.104863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:20.104937 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:20.105099 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:20.105101 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:20.105331 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:20.105388 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.105390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.106077 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.106078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.106616 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.106618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.106975 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.106976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.107032 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.107492 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:20.107497 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:20.109259 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":100,"threads_started":1}
W20260927 18:44:20.109438 24947 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:20.104745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:32775 (local address 127.15.110.254:41719)
0927 18:44:20.104839 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:20.104842 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:20.104925 (+ 83us) server_negotiation.cc:407] Connection header received
0927 18:44:20.105002 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:20.105002 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:20.105024 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:20.105092 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:20.105464 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.105988 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.106703 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.106899 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.106976 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.109359 (+ 2383us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:32775: BlockingRecv error: recv got EOF from 127.15.110.195:32775 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:20.826717 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:20.821986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:37059)
0927 18:44:20.822135 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:20.822146 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:20.822196 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:20.822381 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:20.822382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:20.822559 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:20.822608 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.822611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.823346 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.823347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.823883 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.823885 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.824250 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.824251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.824312 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.824915 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:20.824919 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:20.826652 (+ 1733us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":83,"threads_started":1}
W20260927 18:44:20.826923 24951 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:20.822106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:37059 (local address 127.15.110.254:41719)
0927 18:44:20.822248 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:20.822250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:20.822258 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:20.822284 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:20.822284 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:20.822297 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:20.822360 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:20.822709 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.823273 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.823963 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.824162 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.824217 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.826820 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:37059: BlockingRecv error: recv got EOF from 127.15.110.193:37059 (error 108)
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":89,"threads_started":1}
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:20 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:20.986810 24952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:20.972008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:52799)
0927 18:44:20.972154 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:20.972166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:20.972231 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:20.972360 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:20.972361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:20.972513 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:20.972562 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.972564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.973203 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.973203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.973735 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:20.973737 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.974178 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.974179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:20.974254 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.984582 (+ 10328us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:20.984588 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:20.986750 (+ 2162us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
W20260927 18:44:20.986986 24951 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:20.972057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:52799 (local address 127.15.110.254:41719)
0927 18:44:20.972130 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:20.972132 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:20.972233 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:44:20.972290 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:20.972291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:20.972303 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:20.972344 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:20.972625 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.973138 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.973812 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:20.974050 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:20.974166 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:20.986907 (+ 12741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:52799: BlockingRecv error: recv got EOF from 127.15.110.194:52799 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:21.114161 24953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:21.109816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:46493)
0927 18:44:21.109959 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:21.109970 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:21.110027 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:21.110138 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:21.110139 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:21.110291 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:21.110339 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.110340 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.110960 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.110961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.111527 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.111529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.111862 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.111863 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.111904 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.112331 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:21.112335 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:21.114100 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":81,"threads_started":1}
W20260927 18:44:21.114297 24951 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:21.109921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:46493 (local address 127.15.110.254:41719)
0927 18:44:21.109987 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:21.109989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:21.110026 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:44:21.110070 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:21.110070 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:21.110083 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:21.110134 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:21.110394 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.110889 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.111592 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.111797 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.111850 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.114214 (+ 2364us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:46493: BlockingRecv error: recv got EOF from 127.15.110.195:46493 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:21.832309 24954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:21.827280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:56743)
0927 18:44:21.827415 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:21.827426 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:21.827487 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:21.827614 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:21.827615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:21.827767 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:21.827816 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.827817 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.828598 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.828600 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.829421 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.829425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.829790 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.829791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.829831 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.830359 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:21.830365 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:21.832251 (+ 1886us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":72,"threads_started":1}
W20260927 18:44:21.832463 24955 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:21.827348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56743 (local address 127.15.110.254:41719)
0927 18:44:21.827497 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:21.827499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:21.827508 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:21.827533 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:21.827533 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:21.827544 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:21.827609 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:21.827877 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.828418 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.829504 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.829721 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.829781 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.832370 (+ 2589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:56743: BlockingRecv error: recv got EOF from 127.15.110.193:56743 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:21 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:21.992302 24956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:21.987323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:58535)
0927 18:44:21.987520 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:21.987534 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:21.987602 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:21.987787 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:21.987788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:21.987952 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:21.988001 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.988003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.988689 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.988690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.989265 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:21.989267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.989662 (+ 395us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.989663 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:21.989713 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.990251 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:21.990256 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:21.992205 (+ 1949us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":109,"threads_started":1}
W20260927 18:44:21.992350 24955 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:21.987536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:58535 (local address 127.15.110.254:41719)
0927 18:44:21.987611 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:21.987613 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:21.987622 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:21.987673 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:21.987674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:21.987685 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:21.987736 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:21.988112 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.988607 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.989360 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:21.989564 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:21.989629 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:21.992285 (+ 2656us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:58535: BlockingRecv error: recv got EOF from 127.15.110.194:58535 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:22.119814 24957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:22.114603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:52867)
0927 18:44:22.114725 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:22.114737 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:22.114795 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:22.114927 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:22.114928 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:22.115152 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:22.115242 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.115245 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.116266 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.116268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.116961 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.116964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.117360 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.117360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.117423 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.117857 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:22.117862 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:22.119756 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":57,"threads_started":1}
W20260927 18:44:22.119956 24955 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:22.114655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:52867 (local address 127.15.110.254:41719)
0927 18:44:22.114699 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:22.114701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:22.114795 (+ 94us) server_negotiation.cc:407] Connection header received
0927 18:44:22.114855 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:22.114855 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:22.114867 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:22.114918 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:22.115352 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.116177 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.117053 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.117271 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.117357 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.119872 (+ 2515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:52867: BlockingRecv error: recv got EOF from 127.15.110.195:52867 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:22.837484 24958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:22.832803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:38525)
0927 18:44:22.832934 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:22.832946 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:22.832999 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:22.833147 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:22.833148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:22.833315 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:22.833365 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.833366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.834081 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.834082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.834658 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.834660 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.834977 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.834978 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.835067 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.835671 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:22.835675 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:22.837426 (+ 1751us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":67,"threads_started":1}
W20260927 18:44:22.837690 24959 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:22.832849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:38525 (local address 127.15.110.254:41719)
0927 18:44:22.832968 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:22.832970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:22.833005 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:44:22.833045 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:22.833046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:22.833056 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:22.833108 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:22.833439 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.833974 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.834723 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.834916 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.834969 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.837624 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:38525: BlockingRecv error: recv got EOF from 127.15.110.193:38525 (error 108)
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":66,"threads_started":1}
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:22 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:22.997694 24960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:22.992786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:44729)
0927 18:44:22.992951 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:22.992973 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:22.993083 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:22.993250 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:22.993251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:22.993434 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:22.993484 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.993486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.994182 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.994183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.994716 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:22.994718 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.995090 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.995091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:22.995157 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.995663 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:22.995668 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:22.997635 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
W20260927 18:44:22.997830 24959 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:22.992906 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:44729 (local address 127.15.110.254:41719)
0927 18:44:22.992981 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:22.992983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:22.993066 (+ 83us) server_negotiation.cc:407] Connection header received
0927 18:44:22.993151 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:22.993151 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:22.993171 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:22.993227 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:22.993593 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.994088 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.994802 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:22.995001 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:22.995090 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:22.997748 (+ 2658us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:44729: BlockingRecv error: recv got EOF from 127.15.110.194:44729 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:23.124724 24961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:23.120308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:42547)
0927 18:44:23.120429 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:23.120444 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:23.120500 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:23.120614 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:23.120615 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:23.120796 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:23.120846 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:23.120847 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.121466 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.121467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:23.121995 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:23.121996 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.122303 (+ 307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.122303 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:23.122341 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:23.122844 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:23.122849 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:23.124658 (+ 1809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":67,"threads_started":1}
W20260927 18:44:23.124886 24959 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:23.120369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42547 (local address 127.15.110.254:41719)
0927 18:44:23.120436 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:23.120438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:23.120502 (+ 64us) server_negotiation.cc:407] Connection header received
0927 18:44:23.120545 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:23.120546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:23.120558 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:23.120609 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:23.120906 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.121400 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.122065 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.122242 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.122291 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:23.124802 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:42547: BlockingRecv error: recv got EOF from 127.15.110.195:42547 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:23.842432 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:23.837960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:56215)
0927 18:44:23.838114 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:23.838125 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:23.838180 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:23.838332 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:23.838333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:23.838491 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:23.838541 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:23.838543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.839275 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.839276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:23.839814 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:23.839816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.840148 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.840149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:23.840193 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:23.840637 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:23.840642 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:23.842372 (+ 1730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
W20260927 18:44:23.842561 24963 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:23.838065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56215 (local address 127.15.110.254:41719)
0927 18:44:23.838205 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:23.838207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:23.838216 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:23.838247 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:23.838247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:23.838258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:23.838314 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:23.838627 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.839128 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.839886 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.840079 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.840131 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:23.842464 (+ 2333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:56215: BlockingRecv error: recv got EOF from 127.15.110.193:56215 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":96,"threads_started":1}
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:23 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:24.003058 24964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:23.998108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:40909)
0927 18:44:23.998318 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:23.998331 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:23.998399 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:23.998538 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:23.998539 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:23.998742 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:23.998790 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:23.998792 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.999525 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:23.999527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.000082 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:24.000083 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.000458 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.000459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.000518 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.000976 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:24.000981 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:24.002984 (+ 2003us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":111,"threads_started":1}
W20260927 18:44:24.003180 24963 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:23.998216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:40909 (local address 127.15.110.254:41719)
0927 18:44:23.998308 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:23.998310 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:23.998379 (+ 69us) server_negotiation.cc:407] Connection header received
0927 18:44:23.998453 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:23.998453 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:23.998467 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:23.998532 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:23.998878 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:23.999449 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.000175 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.000374 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.000458 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.003101 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:40909: BlockingRecv error: recv got EOF from 127.15.110.194:40909 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
Sep 27 18:44:24 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:24 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:24.130020 24965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:24.125274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:49371)
0927 18:44:24.125431 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:24.125443 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:24.125508 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:24.125652 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:24.125653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:24.125819 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:24.125868 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:24.125869 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.126543 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.126544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.127089 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:24.127091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.127483 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.127484 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.127531 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.128002 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:24.128006 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:24.129959 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":103,"threads_started":1}
W20260927 18:44:24.130153 24963 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:24.125347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49371 (local address 127.15.110.254:41719)
0927 18:44:24.125426 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:24.125428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:24.125497 (+ 69us) server_negotiation.cc:407] Connection header received
0927 18:44:24.125570 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:24.125571 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:24.125583 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:24.125626 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:24.125934 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.126444 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.127210 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.127396 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.127457 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.130069 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:49371: BlockingRecv error: recv got EOF from 127.15.110.195:49371 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Sep 27 18:44:24 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:24 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:24.847643 24966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:24.842990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:39557)
0927 18:44:24.843137 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:24.843150 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:24.843227 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:24.843372 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:24.843372 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:24.843525 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:24.843626 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:24.843628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.844322 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.844323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.844878 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:24.844880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.845202 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.845203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:24.845251 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.845691 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:24.845695 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:24.847582 (+ 1887us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":88,"threads_started":1}
W20260927 18:44:24.847743 24967 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:24.843108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39557 (local address 127.15.110.254:41719)
0927 18:44:24.843230 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:24.843232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:24.843241 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:24.843290 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:24.843290 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:24.843300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:24.843355 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:24.843731 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.844227 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.844952 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:24.845138 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:24.845187 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:24.847673 (+ 2486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:39557: BlockingRecv error: recv got EOF from 127.15.110.193:39557 (error 108)
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":95,"threads_started":1}
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:25.009209 24968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:25.003636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:54745)
0927 18:44:25.003877 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:25.003887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:25.003958 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:25.004139 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:25.004140 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:25.004355 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:25.004404 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.004406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.005240 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.005241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.006168 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.006171 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.006649 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.006650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.006708 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.007246 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:25.007252 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:25.009111 (+ 1859us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":124,"threads_started":1}
W20260927 18:44:25.009260 24967 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:25.003737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54745 (local address 127.15.110.254:41719)
0927 18:44:25.003818 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:25.003820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:25.004003 (+ 183us) server_negotiation.cc:407] Connection header received
0927 18:44:25.004031 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:25.004032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:25.004044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:25.004094 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:25.004484 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.005163 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.006246 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.006542 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.006648 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.009195 (+ 2547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:54745: BlockingRecv error: recv got EOF from 127.15.110.194:54745 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:25.135645 24969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:25.130529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:47017)
0927 18:44:25.130701 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:25.130722 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:25.130773 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:25.130903 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:25.130904 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:25.131070 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:25.131146 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.131148 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.131882 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.131883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.132428 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.132429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.132834 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.132836 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.132887 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.133351 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:25.133357 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:25.135518 (+ 2161us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":124,"threads_started":1}
W20260927 18:44:25.135775 24967 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:25.130643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:47017 (local address 127.15.110.254:41719)
0927 18:44:25.130740 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:25.130742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:25.130757 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:44:25.130832 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:25.130833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:25.130845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:25.130898 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:25.131287 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.131803 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.132529 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.132719 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.132783 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.135687 (+ 2904us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:47017: BlockingRecv error: recv got EOF from 127.15.110.195:47017 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:25 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:25.852816 24970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:25.848362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:43019)
0927 18:44:25.848504 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:25.848516 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:25.848584 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:25.848753 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:25.848754 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:25.848917 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:25.848965 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.848967 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.849585 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.849585 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.850109 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:25.850110 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.850432 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.850433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:25.850473 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.850932 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:25.850936 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:25.852748 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":79,"threads_started":1}
W20260927 18:44:25.852952 24971 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:25.848411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:43019 (local address 127.15.110.254:41719)
0927 18:44:25.848554 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:25.848556 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:25.848564 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:25.848641 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:25.848642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:25.848652 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:25.848731 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:25.849025 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.849520 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.850173 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:25.850367 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:25.850418 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:25.852843 (+ 2425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:43019: BlockingRecv error: recv got EOF from 127.15.110.193:43019 (error 108)
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":79,"threads_started":1}
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:26.020921 24972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:26.009787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:45245)
0927 18:44:26.009968 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:26.009980 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:26.010072 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:26.010228 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:26.010228 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:26.010416 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:26.010462 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.010464 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.011131 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.011133 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.011846 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.011849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.012204 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.012205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.012261 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.018446 (+ 6185us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:26.018453 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:26.020841 (+ 2388us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":94,"threads_started":1}
W20260927 18:44:26.021049 24971 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:26.009874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:45245 (local address 127.15.110.254:41719)
0927 18:44:26.009952 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:26.009955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:26.010055 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:44:26.010133 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:26.010134 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:26.010147 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:26.010222 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:26.010542 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.011047 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.011929 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.012131 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.012207 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.020966 (+ 8759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:45245: BlockingRecv error: recv got EOF from 127.15.110.194:45245 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:26.140811 24973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:26.136135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:50943)
0927 18:44:26.136276 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:26.136296 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:26.136352 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:26.136495 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:26.136496 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:26.136660 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:26.136708 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.136709 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.137426 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.137427 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.137947 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.137948 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.138321 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.138321 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.138361 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.138864 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:26.138868 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:26.140750 (+ 1882us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
W20260927 18:44:26.140930 24971 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:26.136227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:50943 (local address 127.15.110.254:41719)
0927 18:44:26.136295 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:26.136297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:26.136351 (+ 54us) server_negotiation.cc:407] Connection header received
0927 18:44:26.136413 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:26.136414 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:26.136426 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:26.136481 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:26.136828 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.137354 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.138065 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.138252 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.138308 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.140853 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:50943: BlockingRecv error: recv got EOF from 127.15.110.195:50943 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:26 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:26.858268 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:26.853332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:42485)
0927 18:44:26.853503 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:26.853533 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:26.853599 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:26.853849 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:26.853850 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:26.854102 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:26.854166 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.854170 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.854853 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.854854 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.855445 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:26.855447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.855804 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.855805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:26.855845 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.856309 (+ 464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:26.856313 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:26.858158 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":107,"threads_started":1}
W20260927 18:44:26.858316 24975 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:26.853441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:42485 (local address 127.15.110.254:41719)
0927 18:44:26.853578 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:26.853580 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:26.853610 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:44:26.853668 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:26.853669 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:26.853682 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:26.853806 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:26.854264 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.854782 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.855526 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:26.855735 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:26.855791 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:26.858242 (+ 2451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:42485: BlockingRecv error: recv got EOF from 127.15.110.193:42485 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:27.026453 24976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:27.021489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:54941)
0927 18:44:27.021685 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:27.021697 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:27.021806 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:27.021976 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:27.021977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:27.022161 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:27.022208 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.022209 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.022925 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.022927 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.023545 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.023547 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.023941 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.023942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.023999 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.024460 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:27.024465 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:27.026350 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":121,"threads_started":1}
W20260927 18:44:27.026521 24975 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:27.021581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:54941 (local address 127.15.110.254:41719)
0927 18:44:27.021668 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:27.021671 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:27.021784 (+ 113us) server_negotiation.cc:407] Connection header received
0927 18:44:27.021894 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:27.021895 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:27.021908 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:27.021966 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:27.022310 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.022834 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.023643 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.023864 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.023943 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.026452 (+ 2509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:54941: BlockingRecv error: recv got EOF from 127.15.110.194:54941 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:27.145983 24977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:27.141368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:35191)
0927 18:44:27.141517 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:27.141532 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:27.141605 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:27.141754 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:27.141755 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:27.141923 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:27.141972 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.141974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.142635 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.142636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.143236 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.143239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.143581 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.143582 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.143626 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.144065 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:27.144070 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:27.145885 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":74,"threads_started":1}
W20260927 18:44:27.146034 24975 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:27.141506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:35191 (local address 127.15.110.254:41719)
0927 18:44:27.141573 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:27.141574 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:27.141586 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:44:27.141670 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:27.141671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:27.141684 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:27.141738 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:27.142038 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.142551 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.143327 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.143516 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.143574 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.145967 (+ 2393us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:35191: BlockingRecv error: recv got EOF from 127.15.110.195:35191 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:27 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:27.863463 24978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:27.858730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:40541)
0927 18:44:27.858870 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:27.858882 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:27.858942 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:27.859075 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:27.859076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:27.859324 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:27.859374 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.859376 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.860007 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.860008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.860555 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:27.860556 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.860899 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.860900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:27.860940 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.861373 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:27.861377 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:27.863366 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":75,"threads_started":1}
W20260927 18:44:27.863509 24979 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:27.858776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:40541 (local address 127.15.110.254:41719)
0927 18:44:27.858891 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:27.858893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:27.858927 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:44:27.858987 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:27.858988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:27.858997 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:27.859054 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:27.859439 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.859941 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.860640 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:27.860832 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:27.860888 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:27.863440 (+ 2552us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:40541: BlockingRecv error: recv got EOF from 127.15.110.193:40541 (error 108)
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":47,"threads_started":1}
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:28.031867 24980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:28.027072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:51243)
0927 18:44:28.027248 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:28.027258 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:28.027319 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:28.027440 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:28.027441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:28.027614 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:28.027660 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.027662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.028302 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.028302 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.028908 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.028910 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.029291 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.029293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.029344 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.029883 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:28.029888 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:28.031798 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":89,"threads_started":1}
W20260927 18:44:28.032009 24979 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:28.027128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:51243 (local address 127.15.110.254:41719)
0927 18:44:28.027218 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:28.027220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:28.027321 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:44:28.027368 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:28.027369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:28.027380 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:28.027428 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:28.027718 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.028221 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.029002 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.029196 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.029254 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.031925 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:51243: BlockingRecv error: recv got EOF from 127.15.110.194:51243 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:28.152146 24981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:28.146531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:42369)
0927 18:44:28.146706 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:28.146718 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:28.146838 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:28.146970 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:28.146971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:28.147160 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:28.147266 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.147269 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.148105 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.148107 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.149004 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.149007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.149388 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.149389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.149450 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.149930 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:28.149934 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:28.152037 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":112,"threads_started":1}
W20260927 18:44:28.152184 24979 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:28.146592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42369 (local address 127.15.110.254:41719)
0927 18:44:28.146667 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:28.146669 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:28.146823 (+ 154us) server_negotiation.cc:407] Connection header received
0927 18:44:28.146897 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:28.146898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:28.146913 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:28.146966 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:28.147346 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.148036 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.149090 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.149306 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.149381 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.152118 (+ 2737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:42369: BlockingRecv error: recv got EOF from 127.15.110.195:42369 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:28 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:28.869123 24982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:28.863926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:50165)
0927 18:44:28.864062 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:28.864074 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:28.864132 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:28.864306 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:28.864307 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:28.864474 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:28.864518 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.864520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.865289 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.865290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.865876 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:28.865878 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.866233 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.866234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:28.866274 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.866886 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:28.866892 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:28.869065 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
W20260927 18:44:28.869275 24983 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:28.864027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:50165 (local address 127.15.110.254:41719)
0927 18:44:28.864180 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:28.864181 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:28.864190 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:28.864212 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:28.864213 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:28.864223 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:28.864291 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:28.864644 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.865194 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.865958 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:28.866165 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:28.866221 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:28.869180 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:50165: BlockingRecv error: recv got EOF from 127.15.110.193:50165 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":95,"threads_started":1}
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:29.037295 24984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:29.032319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:34519)
0927 18:44:29.032464 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:29.032478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:29.032564 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:29.032716 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:29.032717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:29.032910 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:29.032955 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.032956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.033706 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.033708 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.034270 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.034272 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.034671 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.034672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.034758 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.035221 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:29.035226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:29.037222 (+ 1996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":88,"threads_started":1}
W20260927 18:44:29.037381 24983 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:29.032439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:34519 (local address 127.15.110.254:41719)
0927 18:44:29.032505 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:29.032507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:29.032541 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:44:29.032620 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:29.032621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:29.032634 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:29.032710 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:29.033036 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.033572 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.034364 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.034585 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.034673 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.037332 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:34519: BlockingRecv error: recv got EOF from 127.15.110.194:34519 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:29.158104 24985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:29.152663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:40345)
0927 18:44:29.152878 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:29.152892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:29.152985 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:29.153147 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:29.153148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:29.153344 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:29.153389 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.153391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.154085 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.154087 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.154645 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.154647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.155019 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.155021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.155088 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.155828 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:29.155833 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:29.158006 (+ 2173us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":105,"threads_started":1}
W20260927 18:44:29.158161 24983 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:29.152805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:40345 (local address 127.15.110.254:41719)
0927 18:44:29.152886 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:29.152889 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:29.152967 (+ 78us) server_negotiation.cc:407] Connection header received
0927 18:44:29.153052 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:29.153053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:29.153067 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:29.153142 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:29.153473 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.153989 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.154734 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.154934 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.155010 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.158100 (+ 3090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:40345: BlockingRecv error: recv got EOF from 127.15.110.195:40345 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:29 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:29.875357 24986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:29.869642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:44815)
0927 18:44:29.869797 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:29.869809 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:29.869869 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:29.870060 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:29.870061 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:29.870291 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:29.870370 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.870373 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.871109 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.871111 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.872138 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:29.872141 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.872535 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.872536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:29.872606 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.873094 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:29.873100 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:29.875244 (+ 2144us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":102,"threads_started":1}
W20260927 18:44:29.875469 24987 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:29.869705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:44815 (local address 127.15.110.254:41719)
0927 18:44:29.869832 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:29.869834 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:29.869852 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:44:29.869938 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:29.869938 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:29.869950 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:29.870025 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:29.870456 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.871029 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.872259 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:29.872453 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:29.872506 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:29.875375 (+ 2869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:44815: BlockingRecv error: recv got EOF from 127.15.110.193:44815 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:30.043066 24988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:30.038041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:34233)
0927 18:44:30.038289 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:30.038301 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:30.038369 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:30.038504 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:30.038504 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:30.038667 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:30.038714 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.038716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.039480 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.039482 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.040042 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.040044 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.040438 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.040439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.040495 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.040988 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:30.040992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:30.043003 (+ 2011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":113,"threads_started":1}
W20260927 18:44:30.043242 24987 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:30.038214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:34233 (local address 127.15.110.254:41719)
0927 18:44:30.038280 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:30.038282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:30.038361 (+ 79us) server_negotiation.cc:407] Connection header received
0927 18:44:30.038418 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:30.038418 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:30.038428 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:30.038482 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:30.038793 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.039387 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.040161 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.040360 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.040441 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.043114 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:34233: BlockingRecv error: recv got EOF from 127.15.110.194:34233 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:30.163724 24989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:30.158792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:53243)
0927 18:44:30.158919 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:30.158930 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:30.158994 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:30.159131 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:30.159132 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:30.159349 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:30.159392 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.159394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.160062 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.160063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.160666 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.160668 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.161047 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.161048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.161109 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.161612 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:30.161618 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:30.163651 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260927 18:44:30.164007 24987 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:30.158843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:53243 (local address 127.15.110.254:41719)
0927 18:44:30.158898 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:30.158900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:30.158994 (+ 94us) server_negotiation.cc:407] Connection header received
0927 18:44:30.159049 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:30.159050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:30.159064 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:30.159117 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:30.159472 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.159976 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.160758 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.160959 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.161046 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.163923 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:53243: BlockingRecv error: recv got EOF from 127.15.110.195:53243 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:30 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:30.881027 24990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:30.875910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:36327)
0927 18:44:30.876071 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:30.876082 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:30.876147 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:30.876346 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:30.876348 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:30.876524 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:30.876576 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.876577 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.877323 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.877324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.877866 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:30.877868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.878215 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.878216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:30.878279 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.878886 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:30.878890 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:30.880969 (+ 2079us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":105,"threads_started":1}
W20260927 18:44:30.881309 24991 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:30.876048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:36327 (local address 127.15.110.254:41719)
0927 18:44:30.876185 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:30.876187 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:30.876195 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:44:30.876220 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:30.876220 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:30.876230 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:30.876300 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:30.876666 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.877239 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.877949 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:30.878142 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:30.878199 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:30.881232 (+ 3033us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:36327: BlockingRecv error: recv got EOF from 127.15.110.193:36327 (error 108)
Metrics: {"server-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:31.053910 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:31.043704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:57645)
0927 18:44:31.043886 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:31.043896 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:31.043987 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:31.044131 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:31.044131 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:31.044320 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:31.044365 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.044366 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.045067 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.045068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.045722 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.045724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.046127 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.046128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.046186 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.051821 (+ 5635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:31.051826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:31.053832 (+ 2006us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
W20260927 18:44:31.054064 24991 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:31.043772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:57645 (local address 127.15.110.254:41719)
0927 18:44:31.043841 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:31.043844 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:31.043966 (+ 122us) server_negotiation.cc:407] Connection header received
0927 18:44:31.044059 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:31.044060 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:31.044075 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:31.044128 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:31.044463 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.044972 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.045811 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.046039 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.046126 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.053978 (+ 7852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:57645: BlockingRecv error: recv got EOF from 127.15.110.194:57645 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.195@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:31.169353 24993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:31.164202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.195:35133)
0927 18:44:31.164354 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:31.164368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:31.164476 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:31.164659 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:31.164660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:31.164835 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:31.164885 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.164887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.165597 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.165598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.166133 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.166135 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.166504 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.166505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.166567 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.167049 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:31.167053 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:31.169252 (+ 2199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":100,"threads_started":1}
W20260927 18:44:31.169402 24991 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:31.164280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:35133 (local address 127.15.110.254:41719)
0927 18:44:31.164359 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:31.164361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:31.164464 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:44:31.164574 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:31.164575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:31.164590 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:31.164639 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:31.164968 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.165499 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.166228 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.166429 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.166507 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.169334 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.195:35133: BlockingRecv error: recv got EOF from 127.15.110.195:35133 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:31 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.193@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:31.886662 24994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:31.881720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.193:58815)
0927 18:44:31.881884 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:31.881895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:31.881950 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:31.882130 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:31.882131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:31.882304 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:31.882356 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.882358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.883101 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.883103 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.883733 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:31.883736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.884088 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.884089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:31.884132 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.884576 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:31.884580 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:31.886582 (+ 2002us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
W20260927 18:44:31.886729 24995 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:31.881791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:58815 (local address 127.15.110.254:41719)
0927 18:44:31.881952 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:31.881953 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:31.881992 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:44:31.882026 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:31.882026 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:31.882038 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:31.882095 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:31.882436 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.882994 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.883811 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:31.884018 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:31.884076 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:31.886677 (+ 2601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.193:58815: BlockingRecv error: recv got EOF from 127.15.110.193:58815 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
Sep 27 18:44:32 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
Sep 27 18:44:32 dist-test-slave-vs05 krb5kdc[23935](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.15.110.194@KRBTEST.COM for oryx/127.15.110.254@KRBTEST.COM, Server not found in Kerberos database
I20260927 18:44:32.059290 24996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.054427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:41719 (local address 127.15.110.194:55215)
0927 18:44:32.054628 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:32.054640 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:32.054740 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:32.054917 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:32.054918 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:32.055110 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:32.055157 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.055158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.055856 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.055857 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.056392 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.056394 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.056767 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.056768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.056826 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.057283 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:32.057288 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:32.059223 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.15.110.254:41719: Server oryx/127.15.110.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
W20260927 18:44:32.059420 24995 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:32.054543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:55215 (local address 127.15.110.254:41719)
0927 18:44:32.054623 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.054626 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.054722 (+ 96us) server_negotiation.cc:407] Connection header received
0927 18:44:32.054835 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.054836 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.054847 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.054910 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.055259 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.055764 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.056480 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.056683 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.056768 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.059323 (+ 2555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.15.110.194:55215: BlockingRecv error: recv got EOF from 127.15.110.194:55215 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260927 18:44:32.083091 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24519
I20260927 18:44:32.087622 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24670
I20260927 18:44:32.092289 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24804
I20260927 18:44:32.096365 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 24448
2026-09-27T18:44:32Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17467 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-27T18:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:32Z Disabled control of system clock
I20260927 18:44:32.124125 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33687
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:33687
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260927 18:44:32.208961 25000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:32.209160 25000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:32.209180 25000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:32.210294 25000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:32.210328 25000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:32.210341 25000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:32.210353 25000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:32.210366 25000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:32.211706 25000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:33687
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:32.211905 25000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:32.212142 25000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:32.217145 25006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.217190 25008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.217319 25000 server_base.cc:1034] running on GCE node
W20260927 18:44:32.217435 25005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.217751 25000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:32.218087 25000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:32.219255 25000 hybrid_clock.cc:648] HybridClock initialized: now 1790534672219246 us; error 45 us; skew 500 ppm
I20260927 18:44:32.220579 25000 webserver.cc:460] Webserver started at http://127.15.110.254:43563/ using document root <none> and password file <none>
I20260927 18:44:32.220906 25000 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:32.220986 25000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:32.221163 25000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:32.222309 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "5a700152f185432cbce4b3c69be8d42c"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "b8d4c807deba11d583aa83e490446e1c"
server_key_iv: "fb3ffb11b4bfbe5f863259ebff1fc967"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.222711 25000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "5a700152f185432cbce4b3c69be8d42c"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "b8d4c807deba11d583aa83e490446e1c"
server_key_iv: "fb3ffb11b4bfbe5f863259ebff1fc967"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.224277 25000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:32.225180 25014 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.225391 25000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:32.225507 25000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "5a700152f185432cbce4b3c69be8d42c"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "b8d4c807deba11d583aa83e490446e1c"
server_key_iv: "fb3ffb11b4bfbe5f863259ebff1fc967"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.225593 25000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:32.244493 25000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:32.244855 25000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:32.245029 25000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:32.249312 25000 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:33687
I20260927 18:44:32.249346 25066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:33687 every 8 connection(s)
I20260927 18:44:32.249809 25000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:32.250597 25067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:32.253333 25067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c: Bootstrap starting.
I20260927 18:44:32.253930 25067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:32.254235 25067 log.cc:824] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:32.254973 25067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c: No bootstrap required, opened a new log
I20260927 18:44:32.256656 25067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } }
I20260927 18:44:32.256793 25067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:32.256814 25067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a700152f185432cbce4b3c69be8d42c, State: Initialized, Role: FOLLOWER
I20260927 18:44:32.256994 25067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } }
I20260927 18:44:32.257091 25067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:32.257130 25067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:32.257201 25067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:32.257201 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25000
I20260927 18:44:32.257294 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:32.257501 15803 external_mini_cluster.cc:1428] Setting key 92fee22df4903bffa980a9ceba6e4436
I20260927 18:44:32.257828 25067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } }
I20260927 18:44:32.257953 25067 leader_election.cc:302] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a700152f185432cbce4b3c69be8d42c; no voters:
I20260927 18:44:32.258183 25067 leader_election.cc:288] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:32.258253 25072 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:32.258448 25072 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [term 1 LEADER]: Becoming Leader. State: Replica: 5a700152f185432cbce4b3c69be8d42c, State: Running, Role: LEADER
I20260927 18:44:32.258455 25067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:32.258615 25072 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } }
I20260927 18:44:32.259145 25074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a700152f185432cbce4b3c69be8d42c. Latest consensus state: current_term: 1 leader_uuid: "5a700152f185432cbce4b3c69be8d42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } } }
I20260927 18:44:32.259308 25073 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a700152f185432cbce4b3c69be8d42c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a700152f185432cbce4b3c69be8d42c" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33687 } } }
I20260927 18:44:32.259342 25074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:32.259439 25073 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:32.259690 25081 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:32.260409 25081 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:32.262010 25081 catalog_manager.cc:1347] Generated new cluster ID: 651f1f5a0be04bf4ac15f1e7a8d43cb7
I20260927 18:44:32.262068 25081 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:32.263955 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.258050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37892 (local address 127.15.110.254:33687)
0927 18:44:32.258360 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.258372 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.258415 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:44:32.258553 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.258562 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.258664 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.258808 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.259140 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.260872 (+ 1732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.261946 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.262409 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.262581 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.263323 (+ 742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:32.263327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:32.263327 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:32.263499 (+ 172us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:32.263539 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:32.263550 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:32.263626 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:32.263708 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:44:32.263786 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":106,"threads_started":1}
I20260927 18:44:32.277748 25081 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:32.278338 25081 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:32.284484 25081 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a700152f185432cbce4b3c69be8d42c: Generated new TSK 0
I20260927 18:44:32.284698 25081 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:32.292145 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--builtin_ntp_servers=127.15.110.212:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:44:32.375239 25091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:32.375448 25091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:32.375470 25091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:32.376675 25091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:32.376722 25091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:32.376748 25091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:32.378114 25091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:32.378365 25091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:32.378681 25091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:32.385052 25096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.385099 25097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.385367 25091 server_base.cc:1034] running on GCE node
W20260927 18:44:32.385069 25099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.385684 25091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:32.385954 25091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:32.387130 25091 hybrid_clock.cc:648] HybridClock initialized: now 1790534672387113 us; error 26 us; skew 500 ppm
I20260927 18:44:32.388501 25091 webserver.cc:460] Webserver started at http://127.15.110.193:41299/ using document root <none> and password file <none>
I20260927 18:44:32.388767 25091 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:32.388841 25091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:32.389003 25091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:32.390125 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "01b6451eb1ef464d88bcb9198955e74e"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "037517d48242a9028cddbad2e547671d"
server_key_iv: "2c15015826695889d7603297d1bee5e2"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.390502 25091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "01b6451eb1ef464d88bcb9198955e74e"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "037517d48242a9028cddbad2e547671d"
server_key_iv: "2c15015826695889d7603297d1bee5e2"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.391997 25091 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:44:32.392841 25105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.393040 25091 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:32.393138 25091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "01b6451eb1ef464d88bcb9198955e74e"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "037517d48242a9028cddbad2e547671d"
server_key_iv: "2c15015826695889d7603297d1bee5e2"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.393224 25091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:32.411379 25091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:32.411726 25091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:32.411902 25091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:32.412163 25091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:32.412483 25091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:32.412539 25091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.412580 25091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:32.412618 25091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.418439 25091 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:45283
I20260927 18:44:32.418517 25218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:45283 every 8 connection(s)
I20260927 18:44:32.418854 25091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:32.424341 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.419737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:53393 (local address 127.15.110.254:33687)
0927 18:44:32.419833 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.419841 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.420100 (+ 259us) server_negotiation.cc:407] Connection header received
0927 18:44:32.420539 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.420540 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.420553 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.420619 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.421272 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.421797 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.422800 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.422995 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.423055 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.424125 (+ 1070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:32.424126 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:32.424126 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:32.424130 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:32.424142 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:32.424146 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:32.424191 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:32.424253 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:44:32.424280 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260927 18:44:32.424568 25221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.419708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33687 (local address 127.15.110.193:53393)
0927 18:44:32.419905 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:32.419947 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:32.420439 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:32.420674 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:32.420677 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:32.420686 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:32.421192 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.421196 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.421893 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.421894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.422706 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.422708 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.423091 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.423093 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.423328 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.423987 (+ 659us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:32.423988 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:32.424019 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:32.424043 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:32.424190 (+ 147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:32.424192 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:32.424195 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:32.424242 (+ 47us) client_negotiation.cc:240] Negotiation successful
0927 18:44:32.424319 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":98,"threads_started":1}
I20260927 18:44:32.425197 25219 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33687
I20260927 18:44:32.425309 25219 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:32.425501 25219 heartbeater.cc:511] Master 127.15.110.254:33687 requested a full tablet report, sending...
I20260927 18:44:32.426103 25031 ts_manager.cc:194] Registered new tserver with Master: 01b6451eb1ef464d88bcb9198955e74e (127.15.110.193:45283)
I20260927 18:44:32.426738 25031 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.193:53393
I20260927 18:44:32.427735 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25091
I20260927 18:44:32.427819 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:32.428021 15803 external_mini_cluster.cc:1428] Setting key 295f3dfea8688328a6f790f8cf6d4d37
I20260927 18:44:32.428962 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--builtin_ntp_servers=127.15.110.212:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:44:32.509672 25222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:32.509908 25222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:32.509945 25222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:32.511082 25222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:32.511142 25222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:32.511181 25222 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:32.512599 25222 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:32.512872 25222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:32.513165 25222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:32.518307 25230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.518476 25222 server_base.cc:1034] running on GCE node
W20260927 18:44:32.518335 25227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.518322 25228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.518782 25222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:32.519044 25222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:32.520232 25222 hybrid_clock.cc:648] HybridClock initialized: now 1790534672520209 us; error 53 us; skew 500 ppm
I20260927 18:44:32.521596 25222 webserver.cc:460] Webserver started at http://127.15.110.194:44393/ using document root <none> and password file <none>
I20260927 18:44:32.521835 25222 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:32.521904 25222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:32.522058 25222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:32.523151 25222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "74b4aa07982c466f80cf1f6fb202a336"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "5e11bda03c9b54fbc22f9d0c483d8145"
server_key_iv: "c707080ea5cd7fb18f7b81f2e1c6c808"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.523582 25222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "74b4aa07982c466f80cf1f6fb202a336"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "5e11bda03c9b54fbc22f9d0c483d8145"
server_key_iv: "c707080ea5cd7fb18f7b81f2e1c6c808"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.524935 25222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:32.525673 25236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.525852 25222 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:32.525949 25222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "74b4aa07982c466f80cf1f6fb202a336"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "5e11bda03c9b54fbc22f9d0c483d8145"
server_key_iv: "c707080ea5cd7fb18f7b81f2e1c6c808"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.526029 25222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:32.560405 25222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:32.560751 25222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:32.560910 25222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:32.561168 25222 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:32.561537 25222 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:32.561590 25222 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.561632 25222 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:32.561659 25222 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.567399 25222 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:45333
I20260927 18:44:32.567440 25349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:45333 every 8 connection(s)
I20260927 18:44:32.567781 25222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:32.573225 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.568512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:58905 (local address 127.15.110.254:33687)
0927 18:44:32.568561 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.568562 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.568870 (+ 308us) server_negotiation.cc:407] Connection header received
0927 18:44:32.569429 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.569431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.569443 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.569497 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.570216 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.570711 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.571812 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.572010 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.572086 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.572927 (+ 841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:32.572929 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:32.572929 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:32.572935 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:32.572952 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:32.572957 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:32.573021 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:32.573114 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:44:32.573145 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:44:32.573469 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.568486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33687 (local address 127.15.110.194:58905)
0927 18:44:32.568735 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:32.568778 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:32.569314 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:32.569580 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:32.569590 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:32.569599 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:32.570105 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.570110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.570824 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.570825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.571684 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.571687 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.572094 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.572095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.572234 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.572786 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:32.572786 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:32.572815 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:32.572836 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:32.573021 (+ 185us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:32.573027 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:32.573030 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:32.573116 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:44:32.573220 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":139,"threads_started":1}
I20260927 18:44:32.574007 25350 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33687
I20260927 18:44:32.574122 25350 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:32.574222 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25222
I20260927 18:44:32.574316 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:32.574358 25350 heartbeater.cc:511] Master 127.15.110.254:33687 requested a full tablet report, sending...
I20260927 18:44:32.574542 15803 external_mini_cluster.cc:1428] Setting key 743b978a16b17ed1e805b7266217ab6f
I20260927 18:44:32.574831 25031 ts_manager.cc:194] Registered new tserver with Master: 74b4aa07982c466f80cf1f6fb202a336 (127.15.110.194:45333)
I20260927 18:44:32.575409 25031 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.194:58905
I20260927 18:44:32.575649 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--builtin_ntp_servers=127.15.110.212:46301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260927 18:44:32.656553 25353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:32.656766 25353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:32.656787 25353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:32.658008 25353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:32.658051 25353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:32.658077 25353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:32.659533 25353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:46301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:33687
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:32.659739 25353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:32.660040 25353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:32.665195 25361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.665266 25358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.665381 25353 server_base.cc:1034] running on GCE node
W20260927 18:44:32.665207 25359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.665779 25353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:32.666041 25353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:32.667248 25353 hybrid_clock.cc:648] HybridClock initialized: now 1790534672667224 us; error 42 us; skew 500 ppm
I20260927 18:44:32.668648 25353 webserver.cc:460] Webserver started at http://127.15.110.195:42777/ using document root <none> and password file <none>
I20260927 18:44:32.668910 25353 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:32.668983 25353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:32.669124 25353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:32.670243 25353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "394b7442981e4576bda82da061d18a79"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "19ec6d70c86ec540cde08420f3c00e8d"
server_key_iv: "7cfad9eb9e4a6bf1f5f35120a69fa723"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.670619 25353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "394b7442981e4576bda82da061d18a79"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "19ec6d70c86ec540cde08420f3c00e8d"
server_key_iv: "7cfad9eb9e4a6bf1f5f35120a69fa723"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.672046 25353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:32.672880 25367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.673084 25353 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:32.673206 25353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "394b7442981e4576bda82da061d18a79"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "19ec6d70c86ec540cde08420f3c00e8d"
server_key_iv: "7cfad9eb9e4a6bf1f5f35120a69fa723"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.673286 25353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:32.692324 25353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:32.692643 25353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:32.692813 25353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:32.693040 25353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:32.693362 25353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:32.693416 25353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.693459 25353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:32.693486 25353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.699306 25353 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:34799
I20260927 18:44:32.699384 25480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:34799 every 8 connection(s)
I20260927 18:44:32.699718 25353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:32.700589 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25353
I20260927 18:44:32.700690 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:32.700881 15803 external_mini_cluster.cc:1428] Setting key 33c6475ae244ef6ae7caae0ad9ea24a7
I20260927 18:44:32.705030 25071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.700509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49349 (local address 127.15.110.254:33687)
0927 18:44:32.700581 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.700583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.700763 (+ 180us) server_negotiation.cc:407] Connection header received
0927 18:44:32.701392 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.701393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.701405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.701456 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.702199 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.702712 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.703732 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.703923 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.704003 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.704846 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:32.704847 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:32.704847 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:32.704851 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:32.704861 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:32.704865 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:32.704907 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:32.704953 (+ 46us) server_negotiation.cc:299] Negotiation successful
0927 18:44:32.704974 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:44:32.705250 25483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.700423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:33687 (local address 127.15.110.195:49349)
0927 18:44:32.700633 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:32.700674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:32.701302 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:32.701520 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:32.701523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:32.701531 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:32.702120 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.702125 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.702793 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.702794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.703593 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:32.703595 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:32.704006 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:32.704007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:32.704112 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:32.704704 (+ 592us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:32.704705 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:32.704740 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:32.704773 (+ 33us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:32.704907 (+ 134us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:32.704910 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:32.704912 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:32.704957 (+ 45us) client_negotiation.cc:240] Negotiation successful
0927 18:44:32.705031 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":88,"threads_started":1}
I20260927 18:44:32.705673 25481 heartbeater.cc:347] Connected to a master server at 127.15.110.254:33687
I20260927 18:44:32.705754 25481 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:32.705927 25481 heartbeater.cc:511] Master 127.15.110.254:33687 requested a full tablet report, sending...
I20260927 18:44:32.706296 25031 ts_manager.cc:194] Registered new tserver with Master: 394b7442981e4576bda82da061d18a79 (127.15.110.195:34799)
I20260927 18:44:32.706686 25031 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.195:49349
I20260927 18:44:32.712867 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:44:32.714781 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25091
W20260927 18:44:32.714789 25071 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:32.714239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.15.110.254:33687)
0927 18:44:32.714300 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.714301 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.714400 (+ 99us) server_negotiation.cc:407] Connection header received
0927 18:44:32.714442 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.714443 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.714453 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.714505 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.714705 (+ 200us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37898: BlockingRecv error: recv got EOF from 127.0.0.1:37898 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:44:32.719297 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25222
I20260927 18:44:32.723588 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25353
I20260927 18:44:32.727906 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25000
2026-09-27T18:44:32Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (630 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-27T18:44:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:32Z Disabled control of system clock
I20260927 18:44:32.754694 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:32807
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:32807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:44:32.836047 25492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:32.836304 25492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:32.836339 25492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:32.837503 25492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:32.837576 25492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:32.837604 25492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:32.837631 25492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:32.837662 25492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:32.839006 25492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:32807
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:32807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:32.839303 25492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:32.839617 25492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:32.844367 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.844394 25497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:32.844381 25500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:32.844739 25492 server_base.cc:1034] running on GCE node
I20260927 18:44:32.844950 25492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:32.845182 25492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:32.846362 25492 hybrid_clock.cc:648] HybridClock initialized: now 1790534672846348 us; error 24 us; skew 500 ppm
I20260927 18:44:32.847620 25492 webserver.cc:460] Webserver started at http://127.15.110.254:43179/ using document root <none> and password file <none>
I20260927 18:44:32.847859 25492 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:32.847929 25492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:32.848083 25492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:32.849150 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "89f7eb8d86974a5bba8f551a081a4f41"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "8defa0b8c8f07f12cdde9dca15408a7d"
server_key_iv: "4c1a54cf69d47780babf1320d02e7d19"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.849514 25492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "89f7eb8d86974a5bba8f551a081a4f41"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "8defa0b8c8f07f12cdde9dca15408a7d"
server_key_iv: "4c1a54cf69d47780babf1320d02e7d19"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.850808 25492 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:32.851634 25506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.851799 25492 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:32.851900 25492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "89f7eb8d86974a5bba8f551a081a4f41"
format_stamp: "Formatted at 2026-09-27 18:44:32 on dist-test-slave-vs05"
server_key: "8defa0b8c8f07f12cdde9dca15408a7d"
server_key_iv: "4c1a54cf69d47780babf1320d02e7d19"
server_key_version: "encryptionkey@0"
I20260927 18:44:32.851977 25492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:32.855242 25492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:32.855517 25492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:32.855667 25492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:32.859405 25492 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:32807
I20260927 18:44:32.859433 25558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:32807 every 8 connection(s)
I20260927 18:44:32.859768 25492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:32.860384 25559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:32.863018 25559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41: Bootstrap starting.
I20260927 18:44:32.863608 25559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:32.863902 25559 log.cc:824] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:32.864566 25559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41: No bootstrap required, opened a new log
I20260927 18:44:32.866114 25559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } }
I20260927 18:44:32.866245 25559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:32.866266 25559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89f7eb8d86974a5bba8f551a081a4f41, State: Initialized, Role: FOLLOWER
I20260927 18:44:32.866426 25559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } }
I20260927 18:44:32.866494 25559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:32.866515 25559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:32.866561 25559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:32.867082 25559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } }
I20260927 18:44:32.867228 25559 leader_election.cc:302] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89f7eb8d86974a5bba8f551a081a4f41; no voters:
I20260927 18:44:32.867408 25559 leader_election.cc:288] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:32.867481 25562 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:32.867648 25559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:32.867667 25562 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [term 1 LEADER]: Becoming Leader. State: Replica: 89f7eb8d86974a5bba8f551a081a4f41, State: Running, Role: LEADER
I20260927 18:44:32.867831 25562 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } }
I20260927 18:44:32.868227 25564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89f7eb8d86974a5bba8f551a081a4f41. Latest consensus state: current_term: 1 leader_uuid: "89f7eb8d86974a5bba8f551a081a4f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } } }
I20260927 18:44:32.868320 25564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:32.868448 25563 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89f7eb8d86974a5bba8f551a081a4f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89f7eb8d86974a5bba8f551a081a4f41" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 32807 } } }
I20260927 18:44:32.868556 25563 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41 [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:32.868710 25571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:32.869020 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25492
I20260927 18:44:32.869127 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:32.869267 25571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:32.869321 15803 external_mini_cluster.cc:1428] Setting key a7c58a92e2da5538e7f4b7e03f6aa057
I20260927 18:44:32.871557 25571 catalog_manager.cc:1347] Generated new cluster ID: ca893fd0de024390981846cdb8caf5df
I20260927 18:44:32.871618 25571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:44:32.875512 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:32.869986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45154 (local address 127.15.110.254:32807)
0927 18:44:32.870164 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:32.870173 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:32.870205 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:44:32.870314 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:32.870321 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:32.870379 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:32.870519 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:32.871331 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:32.871335 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:32.871335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:32.871339 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:32.871377 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:32.871391 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:32.871461 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:32.875311 (+ 3850us) server_negotiation.cc:299] Negotiation successful
0927 18:44:32.875368 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":77,"threads_started":1}
I20260927 18:44:32.899305 25571 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:32.899528 25571 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:32.921314 25571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89f7eb8d86974a5bba8f551a081a4f41: Generated new TSK 0
I20260927 18:44:32.921567 25571 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:44:32.931864 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--builtin_ntp_servers=127.15.110.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:44:33.015244 25583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:33.015457 25583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:33.015480 25583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:33.016741 25583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:33.016778 25583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:33.016805 25583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:33.018180 25583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:33.018435 25583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:33.018733 25583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:33.023846 25588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.023860 25591 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.023865 25589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:33.023921 25583 server_base.cc:1034] running on GCE node
I20260927 18:44:33.024292 25583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:33.024505 25583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:33.025686 25583 hybrid_clock.cc:648] HybridClock initialized: now 1790534673025673 us; error 30 us; skew 500 ppm
I20260927 18:44:33.026994 25583 webserver.cc:460] Webserver started at http://127.15.110.193:42307/ using document root <none> and password file <none>
I20260927 18:44:33.027315 25583 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:33.027387 25583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:33.027524 25583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:33.028604 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "4c8695e71ff44f12bc980a5cb86873fe"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "89b9453de296c375d2bd1a4e6032d091"
server_key_iv: "16bdbb12482cf4aad33a3f5538ed055f"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.028975 25583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "4c8695e71ff44f12bc980a5cb86873fe"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "89b9453de296c375d2bd1a4e6032d091"
server_key_iv: "16bdbb12482cf4aad33a3f5538ed055f"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.030310 25583 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:33.031265 25597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.031457 25583 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:33.031562 25583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "4c8695e71ff44f12bc980a5cb86873fe"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "89b9453de296c375d2bd1a4e6032d091"
server_key_iv: "16bdbb12482cf4aad33a3f5538ed055f"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.031656 25583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:33.035148 25583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:33.035444 25583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:33.035588 25583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:33.035823 25583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:33.036185 25583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:33.036232 25583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.036267 25583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:33.036299 25583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.042289 25583 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46507
I20260927 18:44:33.042333 25710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46507 every 8 connection(s)
I20260927 18:44:33.042682 25583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:33.045773 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.043509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39715 (local address 127.15.110.254:32807)
0927 18:44:33.043570 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:33.043572 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:33.043858 (+ 286us) server_negotiation.cc:407] Connection header received
0927 18:44:33.044374 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:33.044375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:33.044383 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:33.044434 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:33.045512 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:33.045513 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:33.045514 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:33.045518 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:33.045535 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:33.045539 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:33.045593 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:33.045684 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 18:44:33.045702 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:44:33.045984 25713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.043499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:32807 (local address 127.15.110.193:39715)
0927 18:44:33.043737 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:33.043771 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:33.044262 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:33.044512 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:33.044515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:33.044523 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:33.045366 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:33.045366 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:33.045394 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:33.045421 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:33.045588 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:33.045592 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:33.045595 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:33.045690 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 18:44:33.045752 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":93,"threads_started":1}
I20260927 18:44:33.046518 25711 heartbeater.cc:347] Connected to a master server at 127.15.110.254:32807
I20260927 18:44:33.046587 25711 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:33.046693 25711 heartbeater.cc:511] Master 127.15.110.254:32807 requested a full tablet report, sending...
I20260927 18:44:33.046964 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25583
I20260927 18:44:33.047055 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:33.047267 15803 external_mini_cluster.cc:1428] Setting key a3936f17c8bce95ff89730644a18fabb
I20260927 18:44:33.047297 25523 ts_manager.cc:194] Registered new tserver with Master: 4c8695e71ff44f12bc980a5cb86873fe (127.15.110.193:46507)
I20260927 18:44:33.048154 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--builtin_ntp_servers=127.15.110.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:44:33.128605 25714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:33.128809 25714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:33.128832 25714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:33.130014 25714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:33.130057 25714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:33.130084 25714 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:33.131470 25714 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:33.131724 25714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:33.132030 25714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:33.137233 25722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.137241 25720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.137243 25719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:33.137707 25714 server_base.cc:1034] running on GCE node
I20260927 18:44:33.137951 25714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:33.138202 25714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:33.139385 25714 hybrid_clock.cc:648] HybridClock initialized: now 1790534673139356 us; error 40 us; skew 500 ppm
I20260927 18:44:33.140728 25714 webserver.cc:460] Webserver started at http://127.15.110.194:40073/ using document root <none> and password file <none>
I20260927 18:44:33.140993 25714 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:33.141074 25714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:33.141232 25714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:33.142320 25714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "e21c988bb6a84fa0a2510dd0a2738080"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "35e3a5cf3bb210517e593af8cfd3759d"
server_key_iv: "4c4be94731dc22898cc61e087ccc468b"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.142701 25714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "e21c988bb6a84fa0a2510dd0a2738080"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "35e3a5cf3bb210517e593af8cfd3759d"
server_key_iv: "4c4be94731dc22898cc61e087ccc468b"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.144110 25714 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:44:33.144873 25728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.145054 25714 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:33.145144 25714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "e21c988bb6a84fa0a2510dd0a2738080"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "35e3a5cf3bb210517e593af8cfd3759d"
server_key_iv: "4c4be94731dc22898cc61e087ccc468b"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.145226 25714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:33.148622 25714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:33.148882 25714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:33.149021 25714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:33.149235 25714 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:33.149528 25714 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:33.149583 25714 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.149622 25714 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:33.149649 25714 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.155827 25714 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:38727
I20260927 18:44:33.155869 25841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:38727 every 8 connection(s)
I20260927 18:44:33.156219 25714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:33.159121 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.156973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:36859 (local address 127.15.110.254:32807)
0927 18:44:33.157041 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:33.157043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:33.157283 (+ 240us) server_negotiation.cc:407] Connection header received
0927 18:44:33.157727 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:33.157728 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:33.157734 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:33.157800 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:33.158902 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:33.158904 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:33.158904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:33.158908 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:33.158925 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:33.158929 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:33.158979 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:33.159038 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 18:44:33.159053 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:44:33.159317 25844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.156967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:32807 (local address 127.15.110.194:36859)
0927 18:44:33.157155 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:33.157195 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:33.157638 (+ 443us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:33.157851 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:33.157855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:33.157862 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:33.158758 (+ 896us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:33.158759 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:33.158786 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:33.158814 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:33.158976 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:33.158979 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:33.158981 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:44:33.159031 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 18:44:33.159077 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":105,"threads_started":1}
I20260927 18:44:33.159767 25842 heartbeater.cc:347] Connected to a master server at 127.15.110.254:32807
I20260927 18:44:33.159852 25842 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:33.159957 25842 heartbeater.cc:511] Master 127.15.110.254:32807 requested a full tablet report, sending...
I20260927 18:44:33.160305 25523 ts_manager.cc:194] Registered new tserver with Master: e21c988bb6a84fa0a2510dd0a2738080 (127.15.110.194:38727)
I20260927 18:44:33.162744 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25714
I20260927 18:44:33.162834 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:33.162973 15803 external_mini_cluster.cc:1428] Setting key 1fc98fe511983a7b547310d2e5f95fb7
I20260927 18:44:33.163960 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--builtin_ntp_servers=127.15.110.212:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260927 18:44:33.245465 25845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:33.245683 25845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:33.245707 25845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:33.246805 25845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:33.246840 25845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:33.246865 25845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:33.248375 25845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:32807
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:33.248565 25845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:33.248862 25845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:33.253925 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.253947 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:33.254015 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:33.254107 25845 server_base.cc:1034] running on GCE node
I20260927 18:44:33.254395 25845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:33.254652 25845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:33.255842 25845 hybrid_clock.cc:648] HybridClock initialized: now 1790534673255827 us; error 30 us; skew 500 ppm
I20260927 18:44:33.257069 25845 webserver.cc:460] Webserver started at http://127.15.110.195:37811/ using document root <none> and password file <none>
I20260927 18:44:33.257313 25845 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:33.257359 25845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:33.257510 25845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:33.258699 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "3a037db4c8c444fc95ef7e1b6bbd5add"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "3ef2df0d3d2c14a1354c76044f340fc0"
server_key_iv: "f6108948baead4625cbeffebb4240f44"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.259059 25845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "3a037db4c8c444fc95ef7e1b6bbd5add"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "3ef2df0d3d2c14a1354c76044f340fc0"
server_key_iv: "f6108948baead4625cbeffebb4240f44"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.260385 25845 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:44:33.261153 25859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.261324 25845 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:44:33.261421 25845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "3a037db4c8c444fc95ef7e1b6bbd5add"
format_stamp: "Formatted at 2026-09-27 18:44:33 on dist-test-slave-vs05"
server_key: "3ef2df0d3d2c14a1354c76044f340fc0"
server_key_iv: "f6108948baead4625cbeffebb4240f44"
server_key_version: "encryptionkey@0"
I20260927 18:44:33.261503 25845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:33.264925 25845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:33.265188 25845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:33.265326 25845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:33.265535 25845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:33.265833 25845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:33.265887 25845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.265926 25845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:33.265954 25845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:33.272284 25845 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:46125
I20260927 18:44:33.272336 25972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:46125 every 8 connection(s)
I20260927 18:44:33.272673 25845 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:44:33.275698 25581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.273450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:55989 (local address 127.15.110.254:32807)
0927 18:44:33.273520 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:33.273522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:33.273780 (+ 258us) server_negotiation.cc:407] Connection header received
0927 18:44:33.274221 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:33.274225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:33.274231 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:33.274284 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:33.275445 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:33.275446 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:33.275447 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:44:33.275450 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:33.275466 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:44:33.275470 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:33.275521 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:33.275633 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 18:44:33.275645 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:44:33.275924 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:33.273441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:32807 (local address 127.15.110.195:55989)
0927 18:44:33.273660 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:33.273693 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:33.274130 (+ 437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:33.274334 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:33.274336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:33.274344 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:33.275296 (+ 952us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:44:33.275296 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:33.275324 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:44:33.275354 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:33.275566 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:33.275569 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:33.275572 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:44:33.275638 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:44:33.275687 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260927 18:44:33.276468 25973 heartbeater.cc:347] Connected to a master server at 127.15.110.254:32807
I20260927 18:44:33.276577 25973 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:33.276736 25973 heartbeater.cc:511] Master 127.15.110.254:32807 requested a full tablet report, sending...
I20260927 18:44:33.277122 25523 ts_manager.cc:194] Registered new tserver with Master: 3a037db4c8c444fc95ef7e1b6bbd5add (127.15.110.195:46125)
I20260927 18:44:33.278710 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25845
I20260927 18:44:33.278829 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:33.279035 15803 external_mini_cluster.cc:1428] Setting key 14d8f52717063e8b1f665c2e651e25ea
I20260927 18:44:33.279966 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260927 18:44:33.281708 25581 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:44:33.281332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45156 (local address 127.15.110.254:32807)
0927 18:44:33.281385 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:33.281387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:33.281452 (+ 65us) server_negotiation.cc:407] Connection header received
0927 18:44:33.281498 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:33.281499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:33.281509 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:33.281568 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:33.281656 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45156: BlockingRecv error: recv got EOF from 127.0.0.1:45156 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:44:33.281780 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25583
I20260927 18:44:33.286917 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25714
I20260927 18:44:33.291100 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25845
I20260927 18:44:33.295065 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25492
2026-09-27T18:44:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (567 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:33 dist-test-slave-vs05 krb5kdc[25983](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:33 dist-test-slave-vs05 krb5kdc[25983](info): set up 2 sockets
Sep 27 18:44:33 dist-test-slave-vs05 krb5kdc[25983](info): commencing operation
krb5kdc: starting...
W20260927 18:44:34.336148 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:44:34.455859 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40925
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:40925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:34.537982 25996 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:34.538228 25996 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:34.538265 25996 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:34.539444 25996 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:44:34.539515 25996 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:34.539539 25996 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:34.539563 25996 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:44:34.539582 25996 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:44:34.541090 25996 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:40925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25996
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:34.541370 25996 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:34.541599 25996 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:34.546547 26001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.546541 26004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.546561 26002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:34.546824 25996 server_base.cc:1034] running on GCE node
I20260927 18:44:34.547052 25996 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:34.547366 25996 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:34.548516 25996 hybrid_clock.cc:648] HybridClock initialized: now 1790534674548500 us; error 30 us; skew 500 ppm
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:34.551158 25996 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:44:34.551748 25996 webserver.cc:460] Webserver started at http://127.15.110.254:36547/ using document root <none> and password file <none>
I20260927 18:44:34.552004 25996 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:34.552055 25996 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:34.552206 25996 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:34.553354 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "c4b6f83fab00f53f46bc8fd153f7152b"
server_key_iv: "89c531f07e9ce4f5b8cebebd192a073b"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.553746 25996 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "c4b6f83fab00f53f46bc8fd153f7152b"
server_key_iv: "89c531f07e9ce4f5b8cebebd192a073b"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.555294 25996 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:34.556227 26011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.556550 25996 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:34.556631 25996 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "c4b6f83fab00f53f46bc8fd153f7152b"
server_key_iv: "89c531f07e9ce4f5b8cebebd192a073b"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.556759 25996 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:34.576006 25996 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:34.578100 25996 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:34.578253 25996 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:34.582208 25996 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:40925
I20260927 18:44:34.582276 26063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:40925 every 8 connection(s)
I20260927 18:44:34.582629 25996 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:44:34.583709 26064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:34.586122 26064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: Bootstrap starting.
I20260927 18:44:34.586721 26064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:34.587006 26064 log.cc:824] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:34.587754 26064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: No bootstrap required, opened a new log
I20260927 18:44:34.589327 26064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } }
I20260927 18:44:34.589488 26064 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:34.589537 26064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52cf9a0066a54b2a815c75b5c3fa8f6a, State: Initialized, Role: FOLLOWER
I20260927 18:44:34.589740 26064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } }
I20260927 18:44:34.589844 26064 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:44:34.589888 26064 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:44:34.589938 26064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:34.590672 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 25996
I20260927 18:44:34.590760 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:44:34.590770 26064 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } }
I20260927 18:44:34.590879 26064 leader_election.cc:302] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52cf9a0066a54b2a815c75b5c3fa8f6a; no voters:
I20260927 18:44:34.590956 15803 external_mini_cluster.cc:1428] Setting key ee9cd215812adf156c96a5fb79dd3f01
I20260927 18:44:34.591059 26064 leader_election.cc:288] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:44:34.591084 26067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:34.591230 26067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [term 1 LEADER]: Becoming Leader. State: Replica: 52cf9a0066a54b2a815c75b5c3fa8f6a, State: Running, Role: LEADER
I20260927 18:44:34.591361 26064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:44:34.591370 26067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } }
I20260927 18:44:34.592032 26069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52cf9a0066a54b2a815c75b5c3fa8f6a. Latest consensus state: current_term: 1 leader_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } } }
I20260927 18:44:34.592133 26069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [sys.catalog]: This master's current role is: LEADER
I20260927 18:44:34.592444 26068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52cf9a0066a54b2a815c75b5c3fa8f6a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40925 } } }
I20260927 18:44:34.592530 26068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a [sys.catalog]: This master's current role is: LEADER
W20260927 18:44:34.593278 26085 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:44:34.593334 26085 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:44:34.593377 26078 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:44:34.593516 26078 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:44:34.595016 26078 catalog_manager.cc:1347] Generated new cluster ID: a534a69fc90e4f62800fc7a5640faea6
I20260927 18:44:34.595068 26078 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:34.603243 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:34.591605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44088 (local address 127.15.110.254:40925)
0927 18:44:34.591799 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:34.591810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:34.591845 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:44:34.591968 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:34.591975 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:34.592032 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:34.592798 (+ 766us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:34.593016 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.594152 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.595171 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.595791 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.595899 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:34.598864 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:34.598869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:34.598869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:34.598881 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:34.601104 (+ 2223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.601505 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.601508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.601508 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.601628 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.601835 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.601836 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.601836 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.602165 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:34.602332 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:34.602852 (+ 520us) server_negotiation.cc:299] Negotiation successful
0927 18:44:34.602994 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":74,"threads_started":1}
I20260927 18:44:34.611835 26078 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:44:34.612579 26078 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:44:34.623970 26078 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: Generated new TSK 0
I20260927 18:44:34.624195 26078 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:44:34.653852 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:34.735525 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:34.735734 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:34.735757 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:34.736908 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:34.736951 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:34.736979 26090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:44:34.738518 26090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:34.738761 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:34.738996 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:34.744061 26096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.744122 26098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.744093 26095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:34.744268 26090 server_base.cc:1034] running on GCE node
I20260927 18:44:34.744493 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:34.744742 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:34.745885 26090 hybrid_clock.cc:648] HybridClock initialized: now 1790534674745863 us; error 34 us; skew 500 ppm
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:34.748389 26090 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:44:34.748759 26090 webserver.cc:460] Webserver started at http://127.15.110.193:46055/ using document root <none> and password file <none>
I20260927 18:44:34.748950 26090 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:34.748991 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:34.749076 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:34.750070 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "7d2b93be3ff441ed997bf9d1c902fc06"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "ff8f569be80cd32b949d23d6d8bc955c"
server_key_iv: "0a844174f5b9ee0628d3ed6a4f2b5ef5"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.750377 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "7d2b93be3ff441ed997bf9d1c902fc06"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "ff8f569be80cd32b949d23d6d8bc955c"
server_key_iv: "0a844174f5b9ee0628d3ed6a4f2b5ef5"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.751814 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:34.752625 26105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.752827 26090 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:34.752912 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "7d2b93be3ff441ed997bf9d1c902fc06"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "ff8f569be80cd32b949d23d6d8bc955c"
server_key_iv: "0a844174f5b9ee0628d3ed6a4f2b5ef5"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.752969 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:34.767059 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:34.769040 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:34.769151 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:34.769332 26090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:34.769641 26090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:34.769676 26090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.769698 26090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:34.769745 26090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.775386 26090 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:43395
I20260927 18:44:34.775470 26218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:43395 every 8 connection(s)
I20260927 18:44:34.775792 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:44:34.778883 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 26090
I20260927 18:44:34.779001 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:44:34.779165 15803 external_mini_cluster.cc:1428] Setting key d5a57cb1c226f901beb709fcf296bf76
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:34.785797 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:34.776537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:57911 (local address 127.15.110.254:40925)
0927 18:44:34.776601 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:34.776603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:34.776819 (+ 216us) server_negotiation.cc:407] Connection header received
0927 18:44:34.777290 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:34.777291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:34.777301 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:34.777363 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:34.778280 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.778845 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.780358 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.780585 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.780679 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:34.782847 (+ 2168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:34.782849 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:34.782849 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:34.782857 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:34.784359 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.784797 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.784798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.784798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.784863 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.785118 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.785119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.785120 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.785256 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:34.785331 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:34.785613 (+ 282us) server_negotiation.cc:299] Negotiation successful
0927 18:44:34.785700 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:44:34.786020 26221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:34.776498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40925 (local address 127.15.110.193:57911)
0927 18:44:34.776677 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:34.776726 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:34.777206 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:34.777421 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:34.777425 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:34.777994 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:34.778184 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:34.778188 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.778970 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.778977 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:34.780265 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:34.780268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.780692 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.780693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:34.780828 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:34.781394 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:34.781402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:34.782746 (+ 1344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:34.784448 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:34.784455 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:34.784457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:34.784716 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:34.784931 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:34.784932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:34.784932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:34.785047 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:34.785332 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:34.785339 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:34.785480 (+ 141us) client_negotiation.cc:769] Sending connection context
0927 18:44:34.785590 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 18:44:34.785765 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260927 18:44:34.786549 26219 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40925
I20260927 18:44:34.786662 26219 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:34.786881 26219 heartbeater.cc:511] Master 127.15.110.254:40925 requested a full tablet report, sending...
I20260927 18:44:34.787459 26027 ts_manager.cc:194] Registered new tserver with Master: 7d2b93be3ff441ed997bf9d1c902fc06 (127.15.110.193:43395)
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:44:34.788084 26027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:57911
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:44:34.806878 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:34.890834 26224 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:34.891023 26224 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:34.891047 26224 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:34.892388 26224 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:34.892431 26224 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:34.892458 26224 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:44:34.893878 26224 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26224
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:34.894060 26224 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:34.894227 26224 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:34.899228 26229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.899247 26232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:34.899230 26230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:34.899274 26224 server_base.cc:1034] running on GCE node
I20260927 18:44:34.899654 26224 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:34.899835 26224 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:34.900995 26224 hybrid_clock.cc:648] HybridClock initialized: now 1790534674900972 us; error 33 us; skew 500 ppm
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:34.903520 26224 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:44:34.904055 26224 webserver.cc:460] Webserver started at http://127.15.110.194:43175/ using document root <none> and password file <none>
I20260927 18:44:34.904301 26224 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:34.904371 26224 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:34.904520 26224 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:34.905532 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "1cc7f04c78d749cd87b2f170db907ad4"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "4bcc270623f02f94fad9ecec4bea2721"
server_key_iv: "a9d3291f2f3377cf782208aafd2ed946"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.905897 26224 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "1cc7f04c78d749cd87b2f170db907ad4"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "4bcc270623f02f94fad9ecec4bea2721"
server_key_iv: "a9d3291f2f3377cf782208aafd2ed946"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.907263 26224 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:34.908095 26239 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.908259 26224 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:44:34.908355 26224 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "1cc7f04c78d749cd87b2f170db907ad4"
format_stamp: "Formatted at 2026-09-27 18:44:34 on dist-test-slave-vs05"
server_key: "4bcc270623f02f94fad9ecec4bea2721"
server_key_iv: "a9d3291f2f3377cf782208aafd2ed946"
server_key_version: "encryptionkey@0"
I20260927 18:44:34.908442 26224 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:34.921253 26224 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:34.923642 26224 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:34.923816 26224 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:34.924064 26224 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:34.924361 26224 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:34.924417 26224 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.924460 26224 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:34.924485 26224 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:34.930642 26224 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:33141
I20260927 18:44:34.930684 26352 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:33141 every 8 connection(s)
I20260927 18:44:34.931010 26224 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:44:34.931998 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 26224
I20260927 18:44:34.932116 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:44:34.932307 15803 external_mini_cluster.cc:1428] Setting key 61e60d2c09da05bed0f3c6c661c00d0b
Sep 27 18:44:34 dist-test-slave-vs05 krb5kdc[25983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534674, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
I20260927 18:44:34.942030 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:34.931831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:56809 (local address 127.15.110.254:40925)
0927 18:44:34.931908 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:34.931910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:34.932117 (+ 207us) server_negotiation.cc:407] Connection header received
0927 18:44:34.932809 (+ 692us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:34.932810 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:34.932823 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:34.932880 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:34.933850 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.934623 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.935777 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.936121 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.936203 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:34.938701 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:34.938704 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:34.938704 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:34.938714 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:34.940394 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.940921 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.940923 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.940923 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.940990 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:34.941306 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:34.941307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:34.941307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:34.941429 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:34.941501 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:34.941802 (+ 301us) server_negotiation.cc:299] Negotiation successful
0927 18:44:34.941904 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:44:34.942144 26355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:34.931815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40925 (local address 127.15.110.194:56809)
0927 18:44:34.931989 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:34.932032 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:34.932704 (+ 672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:34.932955 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:34.932959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:34.933550 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:34.933744 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:34.933750 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.934761 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.934762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:34.935647 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:34.935650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:34.936230 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:34.936231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:34.936361 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:34.936932 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:34.936940 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:34.938592 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:34.940496 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:34.940504 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:34.940507 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:34.940821 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:34.941066 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:34.941067 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:34.941068 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:34.941206 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:34.941501 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:34.941505 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:34.941625 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 18:44:34.941763 (+ 138us) client_negotiation.cc:240] Negotiation successful
0927 18:44:34.941957 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260927 18:44:34.942764 26353 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40925
I20260927 18:44:34.942874 26353 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:34.943059 26353 heartbeater.cc:511] Master 127.15.110.254:40925 requested a full tablet report, sending...
I20260927 18:44:34.943517 26027 ts_manager.cc:194] Registered new tserver with Master: 1cc7f04c78d749cd87b2f170db907ad4 (127.15.110.194:33141)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
I20260927 18:44:34.943974 26027 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:56809
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:44:34.958959 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:44:35.042590 26358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:44:35.042836 26358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:44:35.042876 26358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:44:35.044137 26358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:44:35.044206 26358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:44:35.044252 26358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:44:35.045766 26358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:40925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:44:35.046046 26358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:44:35.046273 26358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:44:35.051429 26364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:44:35.051424 26363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:35.051666 26358 server_base.cc:1034] running on GCE node
W20260927 18:44:35.051455 26366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:44:35.051903 26358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:44:35.052096 26358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:44:35.053267 26358 hybrid_clock.cc:648] HybridClock initialized: now 1790534675053253 us; error 25 us; skew 500 ppm
Sep 27 18:44:35 dist-test-slave-vs05 krb5kdc[25983](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:44:35.055853 26358 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:44:35.056344 26358 webserver.cc:460] Webserver started at http://127.15.110.195:42315/ using document root <none> and password file <none>
I20260927 18:44:35.056566 26358 fs_manager.cc:359] Metadata directory not provided
I20260927 18:44:35.056633 26358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:44:35.056775 26358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:44:35.057812 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8"
format_stamp: "Formatted at 2026-09-27 18:44:35 on dist-test-slave-vs05"
server_key: "a68982eaabd73c1184e1d7a45d00d799"
server_key_iv: "14aefe2e20179e907b1971b2742ba8e1"
server_key_version: "encryptionkey@0"
I20260927 18:44:35.058182 26358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8"
format_stamp: "Formatted at 2026-09-27 18:44:35 on dist-test-slave-vs05"
server_key: "a68982eaabd73c1184e1d7a45d00d799"
server_key_iv: "14aefe2e20179e907b1971b2742ba8e1"
server_key_version: "encryptionkey@0"
I20260927 18:44:35.059669 26358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:44:35.060444 26373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:35.060772 26358 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:44:35.060858 26358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8"
format_stamp: "Formatted at 2026-09-27 18:44:35 on dist-test-slave-vs05"
server_key: "a68982eaabd73c1184e1d7a45d00d799"
server_key_iv: "14aefe2e20179e907b1971b2742ba8e1"
server_key_version: "encryptionkey@0"
I20260927 18:44:35.060941 26358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:44:35.084664 26358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:44:35.086910 26358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:44:35.087064 26358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:44:35.087353 26358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:44:35.087683 26358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:44:35.087740 26358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:35.087772 26358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:44:35.087793 26358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:44:35.093963 26358 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:37983
I20260927 18:44:35.094009 26486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:37983 every 8 connection(s)
I20260927 18:44:35.094362 26358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:44:35 dist-test-slave-vs05 krb5kdc[25983](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534675, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:44:35.104108 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 26358
I20260927 18:44:35.104219 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:44:35.104274 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.095103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:58955 (local address 127.15.110.254:40925)
0927 18:44:35.095148 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.095150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.095404 (+ 254us) server_negotiation.cc:407] Connection header received
0927 18:44:35.096141 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.096142 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.096155 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.096227 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:35.097234 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.097965 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.098977 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.099160 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.099265 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.101470 (+ 2205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:35.101471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:35.101472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:35.101478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:35.102809 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:35.103225 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:35.103227 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:35.103227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:35.103291 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:35.103573 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:35.103574 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:35.103574 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:35.103727 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:35.103809 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.104095 (+ 286us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.104180 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:44:35.104437 15803 external_mini_cluster.cc:1428] Setting key 8ca3a8c081fd163baecbfd8e772afdb3
I20260927 18:44:35.104538 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.095090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:40925 (local address 127.15.110.195:58955)
0927 18:44:35.095283 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.095321 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.096015 (+ 694us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.096314 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.096319 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.096926 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:44:35.097130 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.097135 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.098062 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.098063 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.098901 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.098903 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.099275 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.099277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.099402 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.099993 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:44:35.100002 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:44:35.101378 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:44:35.102885 (+ 1507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:35.102891 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:35.102893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:35.103115 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:35.103379 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:44:35.103380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:44:35.103381 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:44:35.103502 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:44:35.103800 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:44:35.103807 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:44:35.103965 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.104073 (+ 108us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.104237 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":112,"threads_started":1}
I20260927 18:44:35.105010 26487 heartbeater.cc:347] Connected to a master server at 127.15.110.254:40925
I20260927 18:44:35.105132 26487 heartbeater.cc:464] Registering TS with master...
I20260927 18:44:35.105319 26487 heartbeater.cc:511] Master 127.15.110.254:40925 requested a full tablet report, sending...
I20260927 18:44:35.105721 26028 ts_manager.cc:194] Registered new tserver with Master: 798ae0c20bd940c4bdc59ebcfd51b4a8 (127.15.110.195:37983)
I20260927 18:44:35.106099 26028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:58955
I20260927 18:44:35.116371 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:44:35.123644 26071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.117766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.15.110.254:40925)
0927 18:44:35.117863 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.117865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.117908 (+ 43us) server_negotiation.cc:407] Connection header received
0927 18:44:35.117945 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.117946 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.117956 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.118005 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:44:35.118536 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.119017 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.119800 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.119985 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.120053 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.121169 (+ 1116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:44:35.121171 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:44:35.121171 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:44:35.121178 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:44:35.122653 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:35.122939 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:35.122939 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:35.122940 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:35.122989 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:44:35.123128 (+ 139us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:44:35.123129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:44:35.123129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:44:35.123275 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:44:35.123348 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.123497 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.123552 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 18:44:35.125447 26028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:44:35.126989 26028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:44:35.133684 26499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:37983 (local address 127.0.0.1:36848)
0927 18:44:35.129937 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.129945 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.129990 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.130267 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.130271 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.130274 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:35.130343 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.130345 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131331 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.131332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.132463 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.132466 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.133354 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.133355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.133435 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.133449 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.133547 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.133602 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":3,"thread_start_us":164,"threads_started":1}
I20260927 18:44:35.133700 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.15.110.195:37983)
0927 18:44:35.129912 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.129917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.129983 (+ 66us) server_negotiation.cc:407] Connection header received
0927 18:44:35.130089 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.130091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.130107 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.130181 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:35.130446 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131137 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.132553 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.133264 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.133347 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.133404 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.133565 (+ 161us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.133605 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":79,"threads_started":1}
I20260927 18:44:35.134007 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33141 (local address 127.0.0.1:44918)
0927 18:44:35.129726 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.129762 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.129830 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.130466 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.130467 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.130468 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:35.130500 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.130502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131851 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.131852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.132812 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.132814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.133762 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.133764 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.133843 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.133848 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.133899 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.133937 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":3,"thread_start_us":95,"threads_started":1}
I20260927 18:44:35.134270 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44918 (local address 127.15.110.194:33141)
0927 18:44:35.129909 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.129913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.129923 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:44:35.130006 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.130008 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.130020 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.130110 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:35.130585 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131736 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.132905 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.133601 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.133675 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.133724 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.134141 (+ 417us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.134190 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":100,"threads_started":1}
I20260927 18:44:35.134481 26421 tablet_service.cc:1467] Processing CreateTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 (DEFAULT_TABLE table=test-table [id=7a6d9513878b42818349e311b677b5e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:35.134871 26421 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b127e6c4f84405abc5794033a23c4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:35.134836 26287 tablet_service.cc:1467] Processing CreateTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 (DEFAULT_TABLE table=test-table [id=7a6d9513878b42818349e311b677b5e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:35.135077 26287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b127e6c4f84405abc5794033a23c4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:35.137446 26504 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Bootstrap starting.
I20260927 18:44:35.137499 26505 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Bootstrap starting.
I20260927 18:44:35.137929 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43395 (local address 127.0.0.1:47156)
0927 18:44:35.129714 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.129759 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.129831 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.130596 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.130597 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.130598 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:35.130629 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.130630 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131891 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.131892 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.132982 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.132985 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.134473 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.134474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.137719 (+ 3245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.137725 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.137786 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.137840 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":68,"threads_started":1}
I20260927 18:44:35.138221 26504 tablet_bootstrap.cc:654] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:35.138226 26505 tablet_bootstrap.cc:654] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:35.138265 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.129552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47156 (local address 127.15.110.193:43395)
0927 18:44:35.130246 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.130250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.130261 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:44:35.130296 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.130298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.130312 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.130387 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:35.130711 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.131805 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.133073 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.134286 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.134359 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.134437 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.138113 (+ 3676us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.138176 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":656,"thread_start_us":84,"threads_started":1}
I20260927 18:44:35.138804 26505 log.cc:824] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:35.138823 26504 log.cc:824] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:35.139106 26153 tablet_service.cc:1467] Processing CreateTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 (DEFAULT_TABLE table=test-table [id=7a6d9513878b42818349e311b677b5e1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:44:35.139511 26153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b127e6c4f84405abc5794033a23c4e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:44:35.139627 26505 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: No bootstrap required, opened a new log
I20260927 18:44:35.139693 26505 ts_tablet_manager.cc:1397] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:44:35.139850 26504 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: No bootstrap required, opened a new log
I20260927 18:44:35.139902 26504 ts_tablet_manager.cc:1397] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:44:35.142009 26505 raft_consensus.cc:348] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.142208 26505 raft_consensus.cc:374] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:35.142208 26508 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Bootstrap starting.
I20260927 18:44:35.142251 26505 raft_consensus.cc:729] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cc7f04c78d749cd87b2f170db907ad4, State: Initialized, Role: FOLLOWER
I20260927 18:44:35.142382 26505 consensus_queue.cc:260] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.142455 26504 raft_consensus.cc:348] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.142632 26504 raft_consensus.cc:374] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:35.142670 26504 raft_consensus.cc:729] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 798ae0c20bd940c4bdc59ebcfd51b4a8, State: Initialized, Role: FOLLOWER
I20260927 18:44:35.142724 26505 ts_tablet_manager.cc:1428] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:35.142738 26353 heartbeater.cc:503] Master 127.15.110.254:40925 was elected leader, sending a full tablet report...
I20260927 18:44:35.142787 26504 consensus_queue.cc:260] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.142879 26508 tablet_bootstrap.cc:654] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Neither blocks nor log segments found. Creating new log.
I20260927 18:44:35.143093 26487 heartbeater.cc:503] Master 127.15.110.254:40925 was elected leader, sending a full tablet report...
I20260927 18:44:35.143482 26508 log.cc:824] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Log is configured to *not* fsync() on all Append() calls
I20260927 18:44:35.143085 26504 ts_tablet_manager.cc:1428] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:44:35.144694 26508 tablet_bootstrap.cc:492] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: No bootstrap required, opened a new log
I20260927 18:44:35.144788 26508 ts_tablet_manager.cc:1397] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260927 18:44:35.146297 26508 raft_consensus.cc:348] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.146432 26508 raft_consensus.cc:374] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:44:35.146456 26508 raft_consensus.cc:729] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d2b93be3ff441ed997bf9d1c902fc06, State: Initialized, Role: FOLLOWER
I20260927 18:44:35.146615 26508 consensus_queue.cc:260] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.146888 26508 ts_tablet_manager.cc:1428] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:44:35.146958 26219 heartbeater.cc:503] Master 127.15.110.254:40925 was elected leader, sending a full tablet report...
W20260927 18:44:35.181691 26354 tablet.cc:2369] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:44:35.276564 26220 tablet.cc:2369] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:35.329205 26510 raft_consensus.cc:482] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:44:35.329396 26510 raft_consensus.cc:504] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.329857 26510 leader_election.cc:288] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cc7f04c78d749cd87b2f170db907ad4 (127.15.110.194:33141), 7d2b93be3ff441ed997bf9d1c902fc06 (127.15.110.193:43395)
I20260927 18:44:35.334398 26501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.330189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:45055 (local address 127.15.110.193:43395)
0927 18:44:35.330242 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.330244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.330397 (+ 153us) server_negotiation.cc:407] Connection header received
0927 18:44:35.330475 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.330476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.330484 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.330522 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:35.330705 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.331430 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.333272 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.333898 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.334004 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.334018 (+ 14us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.334275 (+ 257us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.334324 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:44:35.334398 26502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.330159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:58895 (local address 127.15.110.194:33141)
0927 18:44:35.330211 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.330213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.330225 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:44:35.330320 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.330321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.330329 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.330409 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:44:35.330611 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.331431 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.333272 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.333966 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.334039 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.334048 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.334275 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.334324 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260927 18:44:35.334396 26489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.330065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33141 (local address 127.15.110.195:58895)
0927 18:44:35.330167 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.330180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.330234 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.330418 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.330419 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.330420 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:35.330504 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.330526 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.331541 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.331542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.333132 (+ 1590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.333136 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.334054 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.334055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.334152 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.334159 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.334232 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.334284 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"mutex_wait_us":150}
I20260927 18:44:35.334403 26513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.330138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43395 (local address 127.15.110.195:45055)
0927 18:44:35.330305 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.330315 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:44:35.330403 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:44:35.330555 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:44:35.330556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:44:35.330558 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:44:35.330622 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.330624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.331541 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.331542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.333138 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:44:35.333140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.334042 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.334043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:44:35.334152 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.334159 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:44:35.334232 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:44:35.334283 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"mutex_wait_us":93,"thread_start_us":96,"threads_started":1}
I20260927 18:44:35.334861 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b127e6c4f84405abc5794033a23c4e1" candidate_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" is_pre_election: true
I20260927 18:44:35.334861 26307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b127e6c4f84405abc5794033a23c4e1" candidate_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc7f04c78d749cd87b2f170db907ad4" is_pre_election: true
I20260927 18:44:35.335035 26173 raft_consensus.cc:2454] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 798ae0c20bd940c4bdc59ebcfd51b4a8 in term 0.
I20260927 18:44:35.335050 26307 raft_consensus.cc:2454] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 798ae0c20bd940c4bdc59ebcfd51b4a8 in term 0.
I20260927 18:44:35.335338 26375 leader_election.cc:302] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 798ae0c20bd940c4bdc59ebcfd51b4a8, 7d2b93be3ff441ed997bf9d1c902fc06; no voters:
I20260927 18:44:35.335502 26510 raft_consensus.cc:2781] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:44:35.335568 26510 raft_consensus.cc:482] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:44:35.335623 26510 raft_consensus.cc:3037] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:35.336449 26510 raft_consensus.cc:504] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.336611 26510 leader_election.cc:288] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [CANDIDATE]: Term 1 election: Requested vote from peers 1cc7f04c78d749cd87b2f170db907ad4 (127.15.110.194:33141), 7d2b93be3ff441ed997bf9d1c902fc06 (127.15.110.193:43395)
I20260927 18:44:35.336799 26307 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b127e6c4f84405abc5794033a23c4e1" candidate_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cc7f04c78d749cd87b2f170db907ad4"
I20260927 18:44:35.336807 26173 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b127e6c4f84405abc5794033a23c4e1" candidate_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7d2b93be3ff441ed997bf9d1c902fc06"
I20260927 18:44:35.336892 26307 raft_consensus.cc:3037] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:35.336889 26173 raft_consensus.cc:3037] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:44:35.337644 26173 raft_consensus.cc:2454] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 798ae0c20bd940c4bdc59ebcfd51b4a8 in term 1.
I20260927 18:44:35.337741 26307 raft_consensus.cc:2454] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 798ae0c20bd940c4bdc59ebcfd51b4a8 in term 1.
I20260927 18:44:35.337846 26375 leader_election.cc:302] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 798ae0c20bd940c4bdc59ebcfd51b4a8, 7d2b93be3ff441ed997bf9d1c902fc06; no voters:
I20260927 18:44:35.337983 26510 raft_consensus.cc:2781] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:44:35.338207 26510 raft_consensus.cc:686] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 1 LEADER]: Becoming Leader. State: Replica: 798ae0c20bd940c4bdc59ebcfd51b4a8, State: Running, Role: LEADER
I20260927 18:44:35.338330 26510 consensus_queue.cc:237] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } }
I20260927 18:44:35.339252 26027 catalog_manager.cc:5495] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 798ae0c20bd940c4bdc59ebcfd51b4a8 (127.15.110.195). New cstate: current_term: 1 leader_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "798ae0c20bd940c4bdc59ebcfd51b4a8" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 37983 } health_report { overall_health: HEALTHY } } }
W20260927 18:44:35.344897 26488 tablet.cc:2369] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:44:35.363863 26503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.361173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36850 (local address 127.15.110.195:37983)
0927 18:44:35.361244 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:44:35.361247 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:44:35.361256 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:44:35.361296 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:44:35.361297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:44:35.361315 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:44:35.361386 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:44:35.361535 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.362045 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.362914 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:44:35.363113 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:44:35.363200 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:44:35.363453 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:44:35.363560 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:44:35.363710 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 18:44:35.363782 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:44:35.363805 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:44:35.365926 26173 raft_consensus.cc:1261] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 1 FOLLOWER]: Refusing update from remote peer 798ae0c20bd940c4bdc59ebcfd51b4a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:35.365979 26307 raft_consensus.cc:1261] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 1 FOLLOWER]: Refusing update from remote peer 798ae0c20bd940c4bdc59ebcfd51b4a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:44:35.366375 26514 consensus_queue.cc:1035] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cc7f04c78d749cd87b2f170db907ad4" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:35.366555 26510 consensus_queue.cc:1035] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7d2b93be3ff441ed997bf9d1c902fc06" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:44:35.369724 26519 mvcc.cc:205] Tried to move back new op lower bound from 7334030030295789568 to 7334030030186450944. Current Snapshot: MvccSnapshot[applied={T|T < 7334030030295789568}]
I20260927 18:44:35.370275 26518 mvcc.cc:205] Tried to move back new op lower bound from 7334030030295789568 to 7334030030186450944. Current Snapshot: MvccSnapshot[applied={T|T < 7334030030295789568 or (T in {7334030030295789568})}]
I20260927 18:44:35.371589 26028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:35.371685 26028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44102:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:44:35.372484 26028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 52cf9a0066a54b2a815c75b5c3fa8f6a: Sending DeleteTablet for 3 replicas of tablet 5b127e6c4f84405abc5794033a23c4e1
I20260927 18:44:35.372818 26153 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:35 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:47156
I20260927 18:44:35.372818 26287 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:35 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:44918
I20260927 18:44:35.372920 26421 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b127e6c4f84405abc5794033a23c4e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:44:35 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36848
I20260927 18:44:35.373153 26528 tablet_replica.cc:330] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: stopping tablet replica
I20260927 18:44:35.373198 26527 tablet_replica.cc:330] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06: stopping tablet replica
I20260927 18:44:35.373245 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 26090
I20260927 18:44:35.373309 26527 raft_consensus.cc:2229] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:35.373420 26527 raft_consensus.cc:2258] T 5b127e6c4f84405abc5794033a23c4e1 P 7d2b93be3ff441ed997bf9d1c902fc06 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:35.373555 26526 tablet_replica.cc:330] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: stopping tablet replica
I20260927 18:44:35.373636 26526 raft_consensus.cc:2229] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:44:35.373725 26526 raft_consensus.cc:2258] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:35.374192 26528 raft_consensus.cc:2229] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:44:35.374322 26528 raft_consensus.cc:2258] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:44:35.374781 26528 ts_tablet_manager.cc:1905] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:35.375514 26526 ts_tablet_manager.cc:1905] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:44:35.377084 26528 ts_tablet_manager.cc:1918] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:35.377228 26528 log.cc:1196] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/5b127e6c4f84405abc5794033a23c4e1
I20260927 18:44:35.377421 26526 ts_tablet_manager.cc:1918] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:44:35.377534 26526 log.cc:1196] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/5b127e6c4f84405abc5794033a23c4e1
I20260927 18:44:35.377699 26528 ts_tablet_manager.cc:1939] T 5b127e6c4f84405abc5794033a23c4e1 P 798ae0c20bd940c4bdc59ebcfd51b4a8: Deleting consensus metadata
I20260927 18:44:35.377833 26526 ts_tablet_manager.cc:1939] T 5b127e6c4f84405abc5794033a23c4e1 P 1cc7f04c78d749cd87b2f170db907ad4: Deleting consensus metadata
I20260927 18:44:35.378219 26012 catalog_manager.cc:4841] TS 798ae0c20bd940c4bdc59ebcfd51b4a8 (127.15.110.195:37983): tablet 5b127e6c4f84405abc5794033a23c4e1 (table test-table [id=7a6d9513878b42818349e311b677b5e1]) successfully deleted
I20260927 18:44:35.378417 26014 catalog_manager.cc:4841] TS 1cc7f04c78d749cd87b2f170db907ad4 (127.15.110.194:33141): tablet 5b127e6c4f84405abc5794033a23c4e1 (table test-table [id=7a6d9513878b42818349e311b677b5e1]) successfully deleted
W20260927 18:44:35.380476 26013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:43395 (error 108)
I20260927 18:44:35.380685 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 26224
I20260927 18:44:35.381476 26500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:44:35.381329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43395 (local address 127.0.0.1:47170)
0927 18:44:35.381404 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 18:44:35.381449 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:43395: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 18:44:35.381673 26013 catalog_manager.cc:4584] TS 7d2b93be3ff441ed997bf9d1c902fc06 (127.15.110.193:43395): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b127e6c4f84405abc5794033a23c4e1: Network error: Client connection negotiation failed: client connection to 127.15.110.193:43395: connect: Connection refused (error 111)
I20260927 18:44:35.385011 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 26358
I20260927 18:44:35.390535 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 25996
2026-09-27T18:44:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2095 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:44:35 dist-test-slave-vs05 krb5kdc[26532](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:44:35 dist-test-slave-vs05 krb5kdc[26532](info): set up 2 sockets
Sep 27 18:44:35 dist-test-slave-vs05 krb5kdc[26532](info): commencing operation
krb5kdc: starting...
W20260927 18:44:36.428397 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:44:36 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534676, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:44:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:44:36Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangeradmin_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangeradmin_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangerlookup_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangerlookup_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
I20260927 18:44:36.593786 15803 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCruPYV/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/postgres -l logfile start
2026-09-27 18:44:39.163 UTC [26562] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:44:39.163 UTC [26562] LOG: listening on IPv4 address "127.15.110.212", port 58211
2026-09-27 18:44:39.167 UTC [26562] LOG: listening on Unix socket "/tmp/.s.PGSQL.58211"
2026-09-27 18:44:39.202 UTC [26565] LOG: database system was shut down at 2026-09-27 18:44:37 UTC
2026-09-27 18:44:39.207 UTC [26562] LOG: database system is ready to accept connections
I20260927 18:44:40.188239 15803 mini_postgres.cc:96] Postgres bound to 58211
2026-09-27 18:44:40.194 UTC [26573] FATAL: database "slave" does not exist
127.15.110.212:58211 - accepting connections
I20260927 18:44:40.194940 15803 mini_ranger.cc:161] Starting Ranger...
I20260927 18:44:40.209474 15803 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:44:40.709272 15803 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:44:40.709357 15803 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin
2026-09-27 18:44:40,997 [I] DB FLAVOR :POSTGRES
2026-09-27 18:44:40,998 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:44:40,998 [I] Checking connection..
2026-09-27 18:44:40,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:44:41,264 [I] Checking connection passed.
2026-09-27 18:44:41,264 [I] --------- Verifying version history table ---------
2026-09-27 18:44:41,264 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:44:41,510 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:44:41,511 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:44:41,761 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:44:41,761 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:44:41,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:44:42.018 UTC [26637] WARNING: there is no transaction in progress
2026-09-27 18:44:42,024 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:44:42,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:44:42,279 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:44:42,279 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:44:42,279 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:44:42,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:44:42,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vs05.c.gcp-upstream.internal','N') ;"
2026-09-27 18:44:42,990 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:44:43,255 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:44:43,255 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:44:43,515 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:44:43,516 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:44:43,760 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:44:44,018 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:44:44,019 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:44:45.005 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.024 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.041 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.057 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.075 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.094 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.158 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.164 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.180 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.191 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.201 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.214 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.224 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.230 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.237 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45.244 UTC [26745] WARNING: there is no transaction in progress
2026-09-27 18:44:45,910 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:44:45,910 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:44:46,165 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:44:46,165 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:44:46,429 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:44:46,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:44:46,694 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:44:46,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:44:46,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:44:47,176 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vs05.c.gcp-upstream.internal';"
2026-09-27 18:44:47,428 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:44:47,428 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:44:47,666 [I] DB_PATCHES have already been applied
I20260927 18:44:47.672782 15803 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCruPYV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:44:47.672852 15803 mini_ranger.cc:193] Using host: 127.15.110.212
I20260927 18:44:47.676030 15803 mini_ranger.cc:227] Ranger admin URL: http://127.15.110.212:36043
Sep 27, 2026 6:44:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:44:47 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:44:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:44:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:44:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:44:48 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:44:48 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:44:48.784602 15803 mini_ranger.cc:160] Time spent starting Ranger: real 8.590s user 0.003s sys 0.005s
Sep 27, 2026 6:45:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:45:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Sep 27, 2026 6:45:09 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:45:09 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:45:10 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:45:10.612795 15803 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangerkms_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/rangerkms_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.110.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.110.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:45:10 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534710, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:45:10.713974 15803 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:45:10.724440 15803 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:45:11.640520 15803 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:45:11.640591 15803 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms
I20260927 18:45:11.640604 15803 mini_ranger_kms.cc:227] Using postgres at 127.15.110.212:58211
2026-09-27 18:45:11,824 [I] DB FLAVOR :POSTGRES
2026-09-27 18:45:11,824 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:45:11,824 [I] Checking connection
2026-09-27 18:45:11,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:45:12,078 [I] connection success
2026-09-27 18:45:12,078 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:45:12,078 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:45:12,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:45:12,332 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:45:12,333 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:45:12,333 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:45:12,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:58211/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:45:12,601 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:45:13.091576 15803 mini_ranger_kms.cc:310] Created kms service
I20260927 18:45:13.282882 15803 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:45:13.324827 15803 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:45:13 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534710, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:13.709316 15803 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:45:13.709448 15803 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:45:13.709473 15803 mini_ranger_kms.cc:240] Using host: 127.15.110.212
I20260927 18:45:13.713115 15803 mini_ranger_kms.cc:276] Ranger KMS PID: 26925
I20260927 18:45:13.713310 15803 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.110.212:54083
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:45:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:45:16.003346 15803 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.289s user 0.014s sys 0.151s
I20260927 18:45:16.003470 15803 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:45:16.003551 15803 mini_ranger_kms.cc:401] 127.15.110.212:54083/kms/v1/keys
Sep 27 18:45:16 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:45:16 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
Sep 27 18:45:17 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534716, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
Sep 27 18:45:19 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:45:19.287257 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:37245
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:37245
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:54083/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:45:19.370091 26976 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:45:19.370301 26976 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:45:19.370328 26976 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:45:19.371536 26976 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:45:19.371578 26976 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:45:19.371593 26976 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:45:19.371605 26976 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:45:19.371647 26976 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:45:19.373168 26976 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.110.212:54083/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:37245
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:37245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26976
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:45:19.373413 26976 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:45:19.373647 26976 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:45:19.379009 26976 server_base.cc:1034] running on GCE node
W20260927 18:45:19.379029 26981 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:19.379019 26982 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:19.379140 26984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:19.379674 26976 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:45:19.379994 26976 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:45:19.381167 26976 hybrid_clock.cc:648] HybridClock initialized: now 1790534719381140 us; error 43 us; skew 500 ppm
Sep 27 18:45:19 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534719, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:45:19.390542 26976 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:45:19.391176 26976 webserver.cc:460] Webserver started at http://127.15.110.254:39179/ using document root <none> and password file <none>
I20260927 18:45:19.391507 26976 fs_manager.cc:359] Metadata directory not provided
I20260927 18:45:19.391582 26976 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:45:19.391727 26976 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:45:19 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534719, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:19.440338 26976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "79ffebc9a5b747b59b25e48e0b317225"
format_stamp: "Formatted at 2026-09-27 18:45:19 on dist-test-slave-vs05"
server_key: "ae66fb125adcbbffe521920f797c1ad3"
server_key_iv: "507b6e5d9baf20ae944d9027ac3402ac"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:19.440776 26976 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "79ffebc9a5b747b59b25e48e0b317225"
format_stamp: "Formatted at 2026-09-27 18:45:19 on dist-test-slave-vs05"
server_key: "ae66fb125adcbbffe521920f797c1ad3"
server_key_iv: "507b6e5d9baf20ae944d9027ac3402ac"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:19.442392 26976 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:45:19.454893 26991 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:19.455112 26976 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:45:19.455236 26976 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "79ffebc9a5b747b59b25e48e0b317225"
format_stamp: "Formatted at 2026-09-27 18:45:19 on dist-test-slave-vs05"
server_key: "ae66fb125adcbbffe521920f797c1ad3"
server_key_iv: "507b6e5d9baf20ae944d9027ac3402ac"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:19.455360 26976 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:45:19.470834 26976 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:45:19.473502 26976 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:45:19.473685 26976 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:45:19.479058 26976 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:37245
I20260927 18:45:19.479079 27043 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:37245 every 8 connection(s)
I20260927 18:45:19.479509 26976 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:45:19.480196 27044 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:45:19.482502 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: Bootstrap starting.
I20260927 18:45:19.482911 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 26976
I20260927 18:45:19.483032 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:45:19.483301 27044 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: Neither blocks nor log segments found. Creating new log.
I20260927 18:45:19.483304 15803 external_mini_cluster.cc:1428] Setting key 844cd13870f691d5cf0bb825535630f9
I20260927 18:45:19.483626 27044 log.cc:824] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: Log is configured to *not* fsync() on all Append() calls
I20260927 18:45:19.484465 27044 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: No bootstrap required, opened a new log
I20260927 18:45:19.487339 27044 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } }
I20260927 18:45:19.487512 27044 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:45:19.487541 27044 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79ffebc9a5b747b59b25e48e0b317225, State: Initialized, Role: FOLLOWER
I20260927 18:45:19.487691 27044 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } }
I20260927 18:45:19.487761 27044 raft_consensus.cc:388] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:45:19.487830 27044 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:45:19.487886 27044 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:45:19.488593 27044 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } }
I20260927 18:45:19.488698 27044 leader_election.cc:302] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 79ffebc9a5b747b59b25e48e0b317225; no voters:
I20260927 18:45:19.488864 27044 leader_election.cc:288] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:45:19.488925 27049 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:45:19.489143 27049 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [term 1 LEADER]: Becoming Leader. State: Replica: 79ffebc9a5b747b59b25e48e0b317225, State: Running, Role: LEADER
I20260927 18:45:19.489248 27044 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:45:19.489272 27049 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } }
I20260927 18:45:19.489681 27049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "79ffebc9a5b747b59b25e48e0b317225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } } }
I20260927 18:45:19.489805 27049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [sys.catalog]: This master's current role is: LEADER
I20260927 18:45:19.489974 27049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79ffebc9a5b747b59b25e48e0b317225. Latest consensus state: current_term: 1 leader_uuid: "79ffebc9a5b747b59b25e48e0b317225" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "79ffebc9a5b747b59b25e48e0b317225" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37245 } } }
I20260927 18:45:19.490064 27049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225 [sys.catalog]: This master's current role is: LEADER
I20260927 18:45:19.490796 27044 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Sep 27 18:45:19 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534719, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:45:19.493835 27058 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:45:19.495126 27058 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:45:19.498564 27058 catalog_manager.cc:1347] Generated new cluster ID: d41428a62e32497886ed5f05066e7cd7
I20260927 18:45:19.498618 27058 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:45:19.499521 27048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:19.483856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49868 (local address 127.15.110.254:37245)
0927 18:45:19.484023 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:19.484033 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:19.484318 (+ 285us) server_negotiation.cc:407] Connection header received
0927 18:45:19.485322 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:19.485331 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:19.485399 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:19.485556 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:45:19.486312 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:19.487825 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:19.488934 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:19.489577 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:19.489685 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:19.493243 (+ 3558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:45:19.493307 (+ 64us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:45:19.493308 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:45:19.493428 (+ 120us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:45:19.496967 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:19.497413 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:19.497417 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:19.497417 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:19.497537 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:19.497776 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:19.497777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:19.497778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:19.498257 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:45:19.498898 (+ 641us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:19.498969 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:45:19.499256 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":91,"threads_started":1}
I20260927 18:45:19.506165 27058 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:45:19.506880 27058 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:45:19.511428 27058 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: Generated new TSK 0
I20260927 18:45:19.511624 27058 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:45:20.226274 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:54083/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:45:20.342906 27089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:45:20.343293 27089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:45:20.343354 27089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:45:20.345633 27089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:45:20.345741 27089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:45:20.345799 27089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:45:20.348203 27089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.110.212:54083/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:45:20.348483 27089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:45:20.348778 27089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:45:20.355520 27094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:20.355751 27097 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:20.355602 27095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:20.356837 27089 server_base.cc:1034] running on GCE node
I20260927 18:45:20.357082 27089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:45:20.357404 27089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:45:20.358585 27089 hybrid_clock.cc:648] HybridClock initialized: now 1790534720358553 us; error 47 us; skew 500 ppm
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:45:20.361676 27089 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:45:20.362224 27089 webserver.cc:460] Webserver started at http://127.15.110.193:45359/ using document root <none> and password file <none>
I20260927 18:45:20.362515 27089 fs_manager.cc:359] Metadata directory not provided
I20260927 18:45:20.362576 27089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:45:20.362710 27089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:20.374905 27089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "9719f4a7dce84c6f9facc66d297b67ef"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "73d8daa670928060811c8f52d2de4f3b"
server_key_iv: "76e4b4eaaed5012c3e3d26338f4f9bab"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.375370 27089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "9719f4a7dce84c6f9facc66d297b67ef"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "73d8daa670928060811c8f52d2de4f3b"
server_key_iv: "76e4b4eaaed5012c3e3d26338f4f9bab"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.377344 27089 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260927 18:45:20.392865 27106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.393129 27089 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:45:20.393230 27089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "9719f4a7dce84c6f9facc66d297b67ef"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "73d8daa670928060811c8f52d2de4f3b"
server_key_iv: "76e4b4eaaed5012c3e3d26338f4f9bab"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.393344 27089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:45:20.415791 27089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:45:20.417888 27089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:45:20.418053 27089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:45:20.418313 27089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:45:20.418694 27089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:45:20.418751 27089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.418805 27089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:45:20.418843 27089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.426355 27089 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:35327
I20260927 18:45:20.426723 27089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:45:20.427186 27219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:35327 every 8 connection(s)
I20260927 18:45:20.434051 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 27089
I20260927 18:45:20.434172 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:45:20.434386 15803 external_mini_cluster.cc:1428] Setting key 59f2f08c5ab8aa4aab36a578f8f46511
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:45:20.441588 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.427626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:53407 (local address 127.15.110.254:37245)
0927 18:45:20.431295 (+ 3669us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.431297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.431325 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:45:20.431865 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.431865 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.431877 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.431935 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:45:20.432653 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.433172 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.434668 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.434949 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.435017 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.438510 (+ 3493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:45:20.438512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:45:20.438512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:45:20.438519 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:45:20.440169 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.440593 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.440594 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.440595 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.440662 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.440923 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.440924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.440924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.441050 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:45:20.441132 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:20.441407 (+ 275us) server_negotiation.cc:299] Negotiation successful
0927 18:45:20.441491 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3637,"thread_start_us":77,"threads_started":1}
I20260927 18:45:20.441777 27222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.427611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:37245 (local address 127.15.110.193:53407)
0927 18:45:20.430991 (+ 3380us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.431048 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.431740 (+ 692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:20.432010 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:20.432015 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:20.432268 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:45:20.432552 (+ 284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.432558 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.433280 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.433281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.434575 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.434578 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.435088 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.435089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.436439 (+ 1350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.436989 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:45:20.436997 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:45:20.438425 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:45:20.440268 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.440275 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.440277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.440502 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.440727 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.440728 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.440729 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.440852 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.441122 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:45:20.441128 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:45:20.441273 (+ 145us) client_negotiation.cc:769] Sending connection context
0927 18:45:20.441386 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:45:20.441529 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3327,"thread_start_us":76,"threads_started":1}
I20260927 18:45:20.442358 27220 heartbeater.cc:347] Connected to a master server at 127.15.110.254:37245
I20260927 18:45:20.442456 27220 heartbeater.cc:464] Registering TS with master...
I20260927 18:45:20.442639 27220 heartbeater.cc:511] Master 127.15.110.254:37245 requested a full tablet report, sending...
I20260927 18:45:20.443219 27008 ts_manager.cc:194] Registered new tserver with Master: 9719f4a7dce84c6f9facc66d297b67ef (127.15.110.193:35327)
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:45:20.444073 27008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:53407
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:45:20.470606 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:54083/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:45:20.567441 27226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:45:20.567660 27226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:45:20.567698 27226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:45:20.569106 27226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:45:20.569176 27226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:45:20.569221 27226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:45:20.570688 27226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.110.212:54083/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:45:20.570942 27226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:45:20.571236 27226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 18:45:20.576800 27233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:20.577260 27236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:20.577447 27226 server_base.cc:1034] running on GCE node
W20260927 18:45:20.577709 27234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:20.577947 27226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:45:20.578228 27226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:45:20.579385 27226 hybrid_clock.cc:648] HybridClock initialized: now 1790534720579363 us; error 34 us; skew 500 ppm
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:45:20.583212 27226 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:45:20.583841 27226 webserver.cc:460] Webserver started at http://127.15.110.194:37871/ using document root <none> and password file <none>
I20260927 18:45:20.584105 27226 fs_manager.cc:359] Metadata directory not provided
I20260927 18:45:20.584172 27226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:45:20.584321 27226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:20.602577 27226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "03912cffbd4541e48044130b7161f330"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "88e2ff6f400d478efc0db0831957e278"
server_key_iv: "a129250b1ef1f51af44e425dfd5bc73b"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.603065 27226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "03912cffbd4541e48044130b7161f330"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "88e2ff6f400d478efc0db0831957e278"
server_key_iv: "a129250b1ef1f51af44e425dfd5bc73b"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.605621 27226 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260927 18:45:20.622761 27243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.622975 27226 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260927 18:45:20.623076 27226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "03912cffbd4541e48044130b7161f330"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "88e2ff6f400d478efc0db0831957e278"
server_key_iv: "a129250b1ef1f51af44e425dfd5bc73b"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.623201 27226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:45:20.644749 27226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:45:20.647325 27226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:45:20.647539 27226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:45:20.647858 27226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:45:20.648272 27226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:45:20.648362 27226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.648437 27226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:45:20.648496 27226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.657382 27357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:46221 every 8 connection(s)
I20260927 18:45:20.657522 27226 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:46221
I20260927 18:45:20.658063 27226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:45:20.658392 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 27226
I20260927 18:45:20.658512 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:45:20.658713 15803 external_mini_cluster.cc:1428] Setting key a2c8d5456a276da4d6279aa9337dc852
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:45:20.675304 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.663350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:37095 (local address 127.15.110.254:37245)
0927 18:45:20.663398 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.663401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.663410 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:45:20.663436 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.663437 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.663450 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.663520 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:45:20.666228 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.667029 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.668607 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.668928 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.668977 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.671743 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:45:20.671745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:45:20.671745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:45:20.671754 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:45:20.673559 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.674103 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.674104 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.674105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.674169 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.674467 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.674468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.674469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.674613 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:45:20.674674 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:20.675069 (+ 395us) server_negotiation.cc:299] Negotiation successful
0927 18:45:20.675168 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:45:20.676134 27360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.659204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:37245 (local address 127.15.110.194:37095)
0927 18:45:20.659486 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.659527 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.660149 (+ 622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:20.665530 (+ 5381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:20.665535 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:20.665802 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:45:20.666098 (+ 296us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.666105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.667155 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.667156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.668494 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.668497 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.669074 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.669075 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.669226 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.669850 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:45:20.669860 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:45:20.671624 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:45:20.673662 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.673668 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.673671 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.674006 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.674241 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.674242 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.674242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.674382 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.674709 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:45:20.674715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:45:20.674900 (+ 185us) client_negotiation.cc:769] Sending connection context
0927 18:45:20.675692 (+ 792us) client_negotiation.cc:240] Negotiation successful
0927 18:45:20.675881 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":77,"threads_started":1}
I20260927 18:45:20.676862 27358 heartbeater.cc:347] Connected to a master server at 127.15.110.254:37245
I20260927 18:45:20.676970 27358 heartbeater.cc:464] Registering TS with master...
I20260927 18:45:20.677168 27358 heartbeater.cc:511] Master 127.15.110.254:37245 requested a full tablet report, sending...
I20260927 18:45:20.677521 27008 ts_manager.cc:194] Registered new tserver with Master: 03912cffbd4541e48044130b7161f330 (127.15.110.194:46221)
I20260927 18:45:20.678009 27008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:37095
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:45:20.706679 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:54083/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:45:20.825896 27363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:45:20.826237 27363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:45:20.826335 27363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:45:20.828152 27363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:45:20.828368 27363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:45:20.828469 27363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:45:20.830822 27363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.110.212:54083/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:37245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:45:20.831318 27363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:45:20.831633 27363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:45:20.837514 27368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:45:20.837917 27371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:20.837869 27363 server_base.cc:1034] running on GCE node
W20260927 18:45:20.838253 27369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:45:20.838546 27363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:45:20.838827 27363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:45:20.840011 27363 hybrid_clock.cc:648] HybridClock initialized: now 1790534720839994 us; error 41 us; skew 500 ppm
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:45:20.847718 27363 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:45:20.848296 27363 webserver.cc:460] Webserver started at http://127.15.110.195:33731/ using document root <none> and password file <none>
I20260927 18:45:20.848596 27363 fs_manager.cc:359] Metadata directory not provided
I20260927 18:45:20.848703 27363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:45:20.848876 27363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:20.862156 27363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "5b5108ded2434afebf798a6df2ea2fcf"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "ae469fae45af46d41ac3030076a0c5eb"
server_key_iv: "7d3fb12fb9e0bd23d8caa4904179eb18"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.862623 27363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "5b5108ded2434afebf798a6df2ea2fcf"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "ae469fae45af46d41ac3030076a0c5eb"
server_key_iv: "7d3fb12fb9e0bd23d8caa4904179eb18"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.867611 27363 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260927 18:45:20.884177 27378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.884454 27363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:45:20.884549 27363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "5b5108ded2434afebf798a6df2ea2fcf"
format_stamp: "Formatted at 2026-09-27 18:45:20 on dist-test-slave-vs05"
server_key: "ae469fae45af46d41ac3030076a0c5eb"
server_key_iv: "7d3fb12fb9e0bd23d8caa4904179eb18"
server_key_version: "kuduclusterkey@0"
I20260927 18:45:20.884641 27363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:45:20.928162 27363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:45:20.931407 27363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:45:20.931636 27363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:45:20.931929 27363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:45:20.932389 27363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:45:20.932483 27363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.932559 27363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:45:20.932621 27363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:45:20.941540 27363 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:36481
I20260927 18:45:20.942065 27363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:45:20.942613 27491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:36481 every 8 connection(s)
I20260927 18:45:20.950012 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 27363
I20260927 18:45:20.950122 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:45:20.950305 15803 external_mini_cluster.cc:1428] Setting key 846cb5846f856cfe30e9292a5c8aefc1
Sep 27 18:45:20 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:45:20.956409 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.943328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:40403 (local address 127.15.110.254:37245)
0927 18:45:20.943374 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.943376 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.944250 (+ 874us) server_negotiation.cc:407] Connection header received
0927 18:45:20.944932 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.944934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.944945 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.944976 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:45:20.945767 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.946533 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.947905 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.948140 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.948180 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.952999 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:45:20.953001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:45:20.953001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:45:20.953010 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:45:20.954754 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.955316 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.955318 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.955318 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.955379 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.955653 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.955654 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.955655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.955794 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:45:20.955846 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:20.956223 (+ 377us) server_negotiation.cc:299] Negotiation successful
0927 18:45:20.956308 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:45:20.957219 27494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.943483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:37245 (local address 127.15.110.195:40403)
0927 18:45:20.944122 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.944166 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.944801 (+ 635us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:20.945156 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:20.945160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:20.945400 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:45:20.945664 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.945670 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.946646 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.946647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.947796 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.947799 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.948272 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.948273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:20.948407 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.949498 (+ 1091us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:45:20.949509 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:45:20.952877 (+ 3368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:45:20.954880 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.954888 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.954890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.955223 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.955449 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:45:20.955449 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:45:20.955450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:45:20.955585 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:45:20.955893 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:45:20.955900 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:45:20.956071 (+ 171us) client_negotiation.cc:769] Sending connection context
0927 18:45:20.956790 (+ 719us) client_negotiation.cc:240] Negotiation successful
0927 18:45:20.956981 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":601,"thread_start_us":89,"threads_started":1}
I20260927 18:45:20.957893 27492 heartbeater.cc:347] Connected to a master server at 127.15.110.254:37245
I20260927 18:45:20.957990 27492 heartbeater.cc:464] Registering TS with master...
I20260927 18:45:20.958187 27492 heartbeater.cc:511] Master 127.15.110.254:37245 requested a full tablet report, sending...
I20260927 18:45:20.958670 27008 ts_manager.cc:194] Registered new tserver with Master: 5b5108ded2434afebf798a6df2ea2fcf (127.15.110.195:36481)
I20260927 18:45:20.959218 27008 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:40403
I20260927 18:45:20.962072 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:45:20.980489 27223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.963896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.15.110.254:37245)
0927 18:45:20.963936 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.963938 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.964183 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:45:20.964276 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.964277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.964287 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.964316 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:45:20.965092 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.965880 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.970712 (+ 4832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.971044 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.971091 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:20.973629 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:45:20.973630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:45:20.973631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:45:20.973639 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:45:20.978564 (+ 4925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.979388 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.979389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.979390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.979486 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:45:20.979805 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:45:20.979806 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:45:20.979806 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:45:20.979966 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:45:20.980057 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:20.980284 (+ 227us) server_negotiation.cc:299] Negotiation successful
0927 18:45:20.980373 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:45:20.987543 27008 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41240:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:45:20.990013 27008 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:45:21.004894 27507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.993410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:35327 (local address 127.0.0.1:58154)
0927 18:45:20.995107 (+ 1697us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.995152 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.995262 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:20.995837 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:20.995840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:20.995842 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:45:20.995934 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.995936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.002332 (+ 6396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.002333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.003495 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.003498 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.004549 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.004550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.004638 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.004654 (+ 16us) client_negotiation.cc:769] Sending connection context
0927 18:45:21.004730 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:45:21.004787 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1665,"thread_start_us":1483,"threads_started":1}
I20260927 18:45:21.006408 27506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.994379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:36481 (local address 127.0.0.1:41124)
0927 18:45:20.996044 (+ 1665us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.996053 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.996095 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:20.996357 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:20.996358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:20.996360 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:45:20.996401 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:20.996402 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.999493 (+ 3091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:20.999494 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.000843 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.000846 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.006151 (+ 5305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.006152 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.006228 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.006233 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:45:21.006288 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:45:21.006331 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1646,"thread_start_us":57,"threads_started":1}
I20260927 18:45:21.006938 27509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.994683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41124 (local address 127.15.110.195:36481)
0927 18:45:20.995418 (+ 735us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.995423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.996148 (+ 725us) server_negotiation.cc:407] Connection header received
0927 18:45:20.996190 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.996193 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.996207 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.996277 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:45:20.996485 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:20.999396 (+ 2911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.000977 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.005969 (+ 4992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.006056 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.006116 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.006791 (+ 675us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.006840 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":78,"threads_started":1}
I20260927 18:45:21.007753 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.993884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58154 (local address 127.15.110.193:35327)
0927 18:45:20.995614 (+ 1730us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:20.995618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:20.995627 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:45:20.995663 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:20.995666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:20.995683 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:20.995756 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:45:20.997090 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.002210 (+ 5120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.003601 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.004368 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.004446 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.004511 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.007632 (+ 3121us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.007682 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1698,"thread_start_us":56,"threads_started":1}
I20260927 18:45:21.008603 27426 tablet_service.cc:1467] Processing CreateTablet for tablet b308c74871944266b85df8eaf596fa6f (DEFAULT_TABLE table=test-table [id=4b785a0bf62845df8e3dd83628dd8912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:45:21.009006 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308c74871944266b85df8eaf596fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:45:21.013962 27154 tablet_service.cc:1467] Processing CreateTablet for tablet b308c74871944266b85df8eaf596fa6f (DEFAULT_TABLE table=test-table [id=4b785a0bf62845df8e3dd83628dd8912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:45:21.014333 27154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308c74871944266b85df8eaf596fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:45:21.022431 27510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.994174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:46221 (local address 127.0.0.1:56554)
0927 18:45:20.995467 (+ 1293us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:20.995476 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:20.995515 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:21.007501 (+ 11986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:21.007503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:21.007505 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:45:21.007542 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.007544 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.020050 (+ 12506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.020051 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.021170 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.021174 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.022192 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.022193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.022260 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.022265 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:45:21.022297 (+ 32us) client_negotiation.cc:240] Negotiation successful
0927 18:45:21.022342 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1277,"thread_start_us":66,"threads_started":1}
I20260927 18:45:21.023088 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:20.994616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56554 (local address 127.15.110.194:46221)
0927 18:45:21.007240 (+ 12624us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:21.007246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:21.007259 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:45:21.007300 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:21.007302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:21.007319 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:21.007402 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:45:21.018725 (+ 11323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.019932 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.021308 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.022084 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.022863 (+ 779us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.022936 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.022966 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.023008 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12575,"thread_start_us":86,"threads_started":1}
I20260927 18:45:21.023435 27512 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Bootstrap starting.
I20260927 18:45:21.024184 27287 tablet_service.cc:1467] Processing CreateTablet for tablet b308c74871944266b85df8eaf596fa6f (DEFAULT_TABLE table=test-table [id=4b785a0bf62845df8e3dd83628dd8912]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:45:21.024526 27287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b308c74871944266b85df8eaf596fa6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:45:21.024344 27512 tablet_bootstrap.cc:654] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Neither blocks nor log segments found. Creating new log.
I20260927 18:45:21.025393 27512 log.cc:824] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Log is configured to *not* fsync() on all Append() calls
I20260927 18:45:21.025506 27511 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Bootstrap starting.
I20260927 18:45:21.026748 27511 tablet_bootstrap.cc:654] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Neither blocks nor log segments found. Creating new log.
I20260927 18:45:21.027395 27511 log.cc:824] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Log is configured to *not* fsync() on all Append() calls
I20260927 18:45:21.027822 27514 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Bootstrap starting.
I20260927 18:45:21.028403 27512 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: No bootstrap required, opened a new log
I20260927 18:45:21.028553 27512 ts_tablet_manager.cc:1397] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260927 18:45:21.028815 27514 tablet_bootstrap.cc:654] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Neither blocks nor log segments found. Creating new log.
I20260927 18:45:21.029098 27514 log.cc:824] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Log is configured to *not* fsync() on all Append() calls
I20260927 18:45:21.031312 27512 raft_consensus.cc:348] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.031581 27512 raft_consensus.cc:374] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:45:21.031673 27512 raft_consensus.cc:729] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9719f4a7dce84c6f9facc66d297b67ef, State: Initialized, Role: FOLLOWER
I20260927 18:45:21.031898 27512 consensus_queue.cc:260] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.032277 27512 ts_tablet_manager.cc:1428] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260927 18:45:21.032323 27220 heartbeater.cc:503] Master 127.15.110.254:37245 was elected leader, sending a full tablet report...
I20260927 18:45:21.034906 27511 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: No bootstrap required, opened a new log
I20260927 18:45:21.035036 27511 ts_tablet_manager.cc:1397] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.002s
I20260927 18:45:21.037154 27511 raft_consensus.cc:348] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.037384 27511 raft_consensus.cc:374] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:45:21.037469 27511 raft_consensus.cc:729] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b5108ded2434afebf798a6df2ea2fcf, State: Initialized, Role: FOLLOWER
I20260927 18:45:21.037662 27511 consensus_queue.cc:260] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.038059 27492 heartbeater.cc:503] Master 127.15.110.254:37245 was elected leader, sending a full tablet report...
I20260927 18:45:21.038025 27511 ts_tablet_manager.cc:1428] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:45:21.039824 27514 tablet_bootstrap.cc:492] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: No bootstrap required, opened a new log
I20260927 18:45:21.039915 27514 ts_tablet_manager.cc:1397] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.000s
I20260927 18:45:21.041810 27514 raft_consensus.cc:348] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.041986 27514 raft_consensus.cc:374] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:45:21.042022 27514 raft_consensus.cc:729] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03912cffbd4541e48044130b7161f330, State: Initialized, Role: FOLLOWER
I20260927 18:45:21.042156 27514 consensus_queue.cc:260] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.042407 27514 ts_tablet_manager.cc:1428] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:45:21.042518 27358 heartbeater.cc:503] Master 127.15.110.254:37245 was elected leader, sending a full tablet report...
Sep 27 18:45:21 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
W20260927 18:45:21.158978 27359 tablet.cc:2369] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:45:21.177960 27221 tablet.cc:2369] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:45:21.194175 27493 tablet.cc:2369] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:45:21 dist-test-slave-vs05 krb5kdc[26532](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534720, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:45:21.459702 27518 raft_consensus.cc:482] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:45:21.459846 27518 raft_consensus.cc:504] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.460256 27518 leader_election.cc:288] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 03912cffbd4541e48044130b7161f330 (127.15.110.194:46221), 9719f4a7dce84c6f9facc66d297b67ef (127.15.110.193:35327)
I20260927 18:45:21.466244 27505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:21.461112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:51771 (local address 127.15.110.193:35327)
0927 18:45:21.461158 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:21.461161 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:21.461483 (+ 322us) server_negotiation.cc:407] Connection header received
0927 18:45:21.461580 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:21.461581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:21.461590 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:21.461617 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:45:21.461968 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.462690 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.465341 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.465867 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.465918 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.465927 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.466129 (+ 202us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.466173 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:45:21.466969 27523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:21.460740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:35327 (local address 127.15.110.195:51771)
0927 18:45:21.461425 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:21.461436 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:21.461512 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:21.461674 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:21.461675 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:21.461677 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:45:21.461883 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.461886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.462802 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.462803 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.465207 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.465210 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.465985 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.465986 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.466051 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.466054 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:45:21.466837 (+ 783us) client_negotiation.cc:240] Negotiation successful
0927 18:45:21.466887 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":40,"threads_started":1}
I20260927 18:45:21.467511 27174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308c74871944266b85df8eaf596fa6f" candidate_uuid: "5b5108ded2434afebf798a6df2ea2fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9719f4a7dce84c6f9facc66d297b67ef" is_pre_election: true
I20260927 18:45:21.467700 27174 raft_consensus.cc:2454] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b5108ded2434afebf798a6df2ea2fcf in term 0.
I20260927 18:45:21.467958 27379 leader_election.cc:302] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b5108ded2434afebf798a6df2ea2fcf, 9719f4a7dce84c6f9facc66d297b67ef; no voters:
I20260927 18:45:21.468091 27518 raft_consensus.cc:2781] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:45:21.468137 27518 raft_consensus.cc:482] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:45:21.468161 27518 raft_consensus.cc:3037] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:45:21.468665 27508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:21.461041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42741 (local address 127.15.110.194:46221)
0927 18:45:21.461104 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:21.461106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:21.461266 (+ 160us) server_negotiation.cc:407] Connection header received
0927 18:45:21.461373 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:21.461374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:21.461386 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:21.461415 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:45:21.461957 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.462780 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.467750 (+ 4970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.468297 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.468345 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.468352 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.468557 (+ 205us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.468595 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:45:21.468902 27522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:21.460540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:46221 (local address 127.15.110.195:42741)
0927 18:45:21.461196 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0927 18:45:21.461209 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:45:21.461296 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:45:21.461792 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:45:21.461793 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:45:21.461795 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:45:21.461871 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.461873 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.466550 (+ 4677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.466552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.467638 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:45:21.467641 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.468411 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.468412 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:45:21.468480 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.468484 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:45:21.468766 (+ 282us) client_negotiation.cc:240] Negotiation successful
0927 18:45:21.468809 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"thread_start_us":75,"threads_started":1}
I20260927 18:45:21.469022 27518 raft_consensus.cc:504] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.469163 27518 leader_election.cc:288] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [CANDIDATE]: Term 1 election: Requested vote from peers 03912cffbd4541e48044130b7161f330 (127.15.110.194:46221), 9719f4a7dce84c6f9facc66d297b67ef (127.15.110.193:35327)
I20260927 18:45:21.469219 27309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308c74871944266b85df8eaf596fa6f" candidate_uuid: "5b5108ded2434afebf798a6df2ea2fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03912cffbd4541e48044130b7161f330" is_pre_election: true
I20260927 18:45:21.469331 27174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308c74871944266b85df8eaf596fa6f" candidate_uuid: "5b5108ded2434afebf798a6df2ea2fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9719f4a7dce84c6f9facc66d297b67ef"
I20260927 18:45:21.469375 27309 raft_consensus.cc:2454] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5b5108ded2434afebf798a6df2ea2fcf in term 0.
I20260927 18:45:21.469403 27174 raft_consensus.cc:3037] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:45:21.469377 27308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b308c74871944266b85df8eaf596fa6f" candidate_uuid: "5b5108ded2434afebf798a6df2ea2fcf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "03912cffbd4541e48044130b7161f330"
I20260927 18:45:21.469609 27308 raft_consensus.cc:3037] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:45:21.470285 27174 raft_consensus.cc:2454] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b5108ded2434afebf798a6df2ea2fcf in term 1.
I20260927 18:45:21.470454 27379 leader_election.cc:302] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5b5108ded2434afebf798a6df2ea2fcf, 9719f4a7dce84c6f9facc66d297b67ef; no voters:
I20260927 18:45:21.470860 27308 raft_consensus.cc:2454] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5b5108ded2434afebf798a6df2ea2fcf in term 1.
I20260927 18:45:21.471339 27518 raft_consensus.cc:2781] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:45:21.471490 27518 raft_consensus.cc:686] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 1 LEADER]: Becoming Leader. State: Replica: 5b5108ded2434afebf798a6df2ea2fcf, State: Running, Role: LEADER
I20260927 18:45:21.471577 27518 consensus_queue.cc:237] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } }
I20260927 18:45:21.472385 27007 catalog_manager.cc:5495] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5b5108ded2434afebf798a6df2ea2fcf (127.15.110.195). New cstate: current_term: 1 leader_uuid: "5b5108ded2434afebf798a6df2ea2fcf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5b5108ded2434afebf798a6df2ea2fcf" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 36481 } health_report { overall_health: HEALTHY } } }
I20260927 18:45:21.556850 27528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:45:21.553138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41138 (local address 127.15.110.195:36481)
0927 18:45:21.553444 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0927 18:45:21.553447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:45:21.553455 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:45:21.553507 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:45:21.553508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:45:21.553532 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:45:21.553613 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:45:21.553811 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.554560 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.555717 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:45:21.556009 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:45:21.556073 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:45:21.556279 (+ 206us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:45:21.556419 (+ 140us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:45:21.556502 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:45:21.556741 (+ 239us) server_negotiation.cc:299] Negotiation successful
0927 18:45:21.556762 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":53,"threads_started":1}
I20260927 18:45:21.560360 27174 raft_consensus.cc:1261] T b308c74871944266b85df8eaf596fa6f P 9719f4a7dce84c6f9facc66d297b67ef [term 1 FOLLOWER]: Refusing update from remote peer 5b5108ded2434afebf798a6df2ea2fcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:45:21.560799 27518 consensus_queue.cc:1035] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "9719f4a7dce84c6f9facc66d297b67ef" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:45:21.571419 27308 raft_consensus.cc:1261] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 1 FOLLOWER]: Refusing update from remote peer 5b5108ded2434afebf798a6df2ea2fcf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:45:21.571887 27518 consensus_queue.cc:1035] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [LEADER]: Connected to new peer: Peer: permanent_uuid: "03912cffbd4541e48044130b7161f330" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 46221 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:45:21.573925 27529 mvcc.cc:205] Tried to move back new op lower bound from 7334030219507986432 to 7334030219148169216. Current Snapshot: MvccSnapshot[applied={T|T < 7334030219507986432}]
I20260927 18:45:21.575762 27534 mvcc.cc:205] Tried to move back new op lower bound from 7334030219507986432 to 7334030219148169216. Current Snapshot: MvccSnapshot[applied={T|T < 7334030219507986432}]
I20260927 18:45:21.576205 27531 mvcc.cc:205] Tried to move back new op lower bound from 7334030219507986432 to 7334030219148169216. Current Snapshot: MvccSnapshot[applied={T|T < 7334030219507986432}]
I20260927 18:45:21.577068 27007 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:45:21.577157 27007 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41240:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:45:21.577984 27007 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79ffebc9a5b747b59b25e48e0b317225: Sending DeleteTablet for 3 replicas of tablet b308c74871944266b85df8eaf596fa6f
I20260927 18:45:21.578315 27154 tablet_service.cc:1514] Processing DeleteTablet for tablet b308c74871944266b85df8eaf596fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:45:21 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:58154
I20260927 18:45:21.578856 27287 tablet_service.cc:1514] Processing DeleteTablet for tablet b308c74871944266b85df8eaf596fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:45:21 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:56554
I20260927 18:45:21.579061 27426 tablet_service.cc:1514] Processing DeleteTablet for tablet b308c74871944266b85df8eaf596fa6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:45:21 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41124
I20260927 18:45:21.579555 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 27089
I20260927 18:45:21.591414 27541 tablet_replica.cc:330] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: stopping tablet replica
I20260927 18:45:21.591630 27541 raft_consensus.cc:2229] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:45:21.591869 27541 raft_consensus.cc:2258] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:45:21.592365 27541 ts_tablet_manager.cc:1905] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:45:21.594110 27542 tablet_replica.cc:330] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: stopping tablet replica
I20260927 18:45:21.594309 27542 raft_consensus.cc:2229] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:45:21.594501 27542 raft_consensus.cc:2258] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:45:21.594719 27541 ts_tablet_manager.cc:1918] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:45:21.594852 27541 log.cc:1196] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/b308c74871944266b85df8eaf596fa6f
I20260927 18:45:21.595008 27542 ts_tablet_manager.cc:1905] T b308c74871944266b85df8eaf596fa6f P 5b5108ded2434afebf798a6df2ea2fcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:45:21.595203 27541 ts_tablet_manager.cc:1939] T b308c74871944266b85df8eaf596fa6f P 03912cffbd4541e48044130b7161f330: Deleting consensus metadata
I20260927 18:45:21.595718 26993 catalog_manager.cc:4841] TS 03912cffbd4541e48044130b7161f330 (127.15.110.194:46221): tablet b308c74871944266b85df8eaf596fa6f (table test-table [id=4b785a0bf62845df8e3dd83628dd8912]) successfully deleted
W20260927 18:45:21.596083 26992 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:35327 (error 108)
I20260927 18:45:21.596720 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 27226
I20260927 18:45:21.604406 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 27363
I20260927 18:45:21.611202 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 26976
I20260927 18:45:21.618265 15803 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:45:21.963164 15803 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:45:21.963362 15803 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:45:22.308256 15803 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:45:22.308 UTC [26562] LOG: received smart shutdown request
2026-09-27 18:45:22.312 UTC [26562] LOG: background worker "logical replication launcher" (PID 26571) exited with exit code 1
2026-09-27 18:45:22.312 UTC [26566] LOG: shutting down
2026-09-27 18:45:22.494 UTC [26562] LOG: database system is shut down
2026-09-27T18:45:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47225 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:45:22 dist-test-slave-vs05 krb5kdc[27554](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:45:22 dist-test-slave-vs05 krb5kdc[27554](info): set up 2 sockets
Sep 27 18:45:22 dist-test-slave-vs05 krb5kdc[27554](info): commencing operation
krb5kdc: starting...
W20260927 18:45:23.667100 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:45:23 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534723, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:45:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:45:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangeradmin_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangeradmin_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangerlookup_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangerlookup_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
I20260927 18:45:23.830395 15803 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskCruPYV/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/postgres -l logfile start
2026-09-27 18:45:26.461 UTC [27584] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-27 18:45:26.461 UTC [27584] LOG: listening on IPv4 address "127.15.110.212", port 46771
2026-09-27 18:45:26.466 UTC [27584] LOG: listening on Unix socket "/tmp/.s.PGSQL.46771"
2026-09-27 18:45:26.504 UTC [27587] LOG: database system was shut down at 2026-09-27 18:45:24 UTC
2026-09-27 18:45:26.509 UTC [27584] LOG: database system is ready to accept connections
I20260927 18:45:27.470997 15803 mini_postgres.cc:96] Postgres bound to 46771
2026-09-27 18:45:27.477 UTC [27595] FATAL: database "slave" does not exist
127.15.110.212:46771 - accepting connections
I20260927 18:45:27.477504 15803 mini_ranger.cc:161] Starting Ranger...
I20260927 18:45:27.492112 15803 mini_ranger.cc:84] Created miniranger Postgres user
I20260927 18:45:28.016027 15803 mini_ranger.cc:87] Created ranger Postgres database
I20260927 18:45:28.016119 15803 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin
2026-09-27 18:45:28,298 [I] DB FLAVOR :POSTGRES
2026-09-27 18:45:28,298 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:45:28,298 [I] Checking connection..
2026-09-27 18:45:28,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-27 18:45:28,530 [I] Checking connection passed.
2026-09-27 18:45:28,530 [I] --------- Verifying version history table ---------
2026-09-27 18:45:28,531 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:45:28,771 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:45:28,771 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:45:29,020 [I] Table x_db_version_h does not exist in database ranger
2026-09-27 18:45:29,020 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-27 18:45:29,020 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-27 18:45:29.279 UTC [27659] WARNING: there is no transaction in progress
2026-09-27 18:45:29,286 [I] create_dbversion_catalog.sql file imported successfully
2026-09-27 18:45:29,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-27 18:45:29,522 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-27 18:45:29,523 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:45:29,523 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-27 18:45:29,786 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-27 18:45:30,059 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vs05.c.gcp-upstream.internal','N') ;"
2026-09-27 18:45:30,307 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:45:30,537 [I] Table x_portal_user does not exist in database ranger
2026-09-27 18:45:30,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:45:30,802 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-27 18:45:30,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:45:31,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:45:31,290 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-27 18:45:31,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-27 18:45:32.357 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.374 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.392 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.409 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.426 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.445 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.512 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.519 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.533 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.543 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.554 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.567 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.577 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.585 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.593 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:32.601 UTC [27767] WARNING: there is no transaction in progress
2026-09-27 18:45:33,328 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-27 18:45:33,328 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-27 18:45:33,567 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-27 18:45:33,567 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-27 18:45:33,808 [I] Table x_portal_user already exists in database 'ranger'
2026-09-27 18:45:33,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-27 18:45:34,072 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-27 18:45:34,073 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-27 18:45:34,324 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-27 18:45:34,554 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vs05.c.gcp-upstream.internal';"
2026-09-27 18:45:34,804 [I] CORE_DB_SCHEMA import status has been updated
2026-09-27 18:45:34,805 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-27 18:45:35,065 [I] DB_PATCHES have already been applied
I20260927 18:45:35.070894 15803 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskCruPYV/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260927 18:45:35.070957 15803 mini_ranger.cc:193] Using host: 127.15.110.212
I20260927 18:45:35.073982 15803 mini_ranger.cc:227] Ranger admin URL: http://127.15.110.212:50635
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-admin/ews/webapp].
Sep 27, 2026 6:45:35 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260927 18:45:36.171878 15803 mini_ranger.cc:160] Time spent starting Ranger: real 8.694s user 0.000s sys 0.009s
Sep 27, 2026 6:45:56 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 27, 2026 6:45:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 27, 2026 6:45:56 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 27, 2026 6:45:56 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 27, 2026 6:45:57 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260927 18:45:57.878042 15803 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.15.110.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.15.110.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangerkms_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.15.110.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/rangerkms_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.15.110.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.15.110.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.15.110.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/HTTP_127.15.110.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 27 18:45:57 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260927 18:45:57.964665 15803 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260927 18:45:57.973950 15803 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260927 18:45:58.903506 15803 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260927 18:45:58.903597 15803 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms
I20260927 18:45:58.903620 15803 mini_ranger_kms.cc:227] Using postgres at 127.15.110.212:46771
2026-09-27 18:45:59,065 [I] DB FLAVOR :POSTGRES
2026-09-27 18:45:59,065 [I] --------- Verifying Ranger DB connection ---------
2026-09-27 18:45:59,065 [I] Checking connection
2026-09-27 18:45:59,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-27 18:45:59,303 [I] connection success
2026-09-27 18:45:59,303 [I] --------- Verifying Ranger DB tables ---------
2026-09-27 18:45:59,303 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-27 18:45:59,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-27 18:45:59,548 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-27 18:45:59,548 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-27 18:45:59,548 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-27 18:45:59,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.15.110.212:46771/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-27 18:45:59,810 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260927 18:46:00.276736 15803 mini_ranger_kms.cc:310] Created kms service
I20260927 18:46:00.493935 15803 mini_ranger_kms.cc:326] Created kudu user
I20260927 18:46:00.539161 15803 mini_ranger_kms.cc:343] Created rangerkms user
Sep 27 18:46:00 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534757, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:00.901297 15803 mini_ranger_kms.cc:384] Added ranger policy
I20260927 18:46:00.901418 15803 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms:/tmp/dist-test-taskCruPYV/build/release/bin/postgresql.jar:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskCruPYV/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260927 18:46:00.901438 15803 mini_ranger_kms.cc:240] Using host: 127.15.110.212
I20260927 18:46:00.905254 15803 mini_ranger_kms.cc:276] Ranger KMS PID: 27947
I20260927 18:46:00.905361 15803 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.15.110.212:33753
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-kms/ews/webapp].
Sep 27, 2026 6:46:02 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260927 18:46:03.241640 15803 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.277s user 0.007s sys 0.138s
I20260927 18:46:03.241750 15803 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260927 18:46:03.241804 15803 mini_ranger_kms.cc:401] 127.15.110.212:33753/kms/v1/keys
Sep 27 18:46:03 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 27 18:46:03 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
Sep 27 18:46:04 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534763, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.15.110.212@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
Sep 27 18:46:06 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:46:06.506320 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46827
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46827
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:06.602608 27998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:06.602838 27998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:06.602862 27998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:06.604096 27998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:06.604138 27998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:06.604153 27998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:06.604166 27998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:06.604178 27998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:06.605703 27998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
--ranger_kms_url=127.15.110.212:33753/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46827
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:06.605950 27998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:06.606197 27998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:06.612226 28004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:06.612244 28006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:06.612438 27998 server_base.cc:1034] running on GCE node
W20260927 18:46:06.612219 28003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:06.612737 27998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:06.613034 27998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:06.614223 27998 hybrid_clock.cc:648] HybridClock initialized: now 1790534766614206 us; error 35 us; skew 500 ppm
Sep 27 18:46:06 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:06.623572 27998 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:06.624131 27998 webserver.cc:460] Webserver started at http://127.15.110.254:41255/ using document root <none> and password file <none>
I20260927 18:46:06.624361 27998 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:06.624423 27998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:06.624588 27998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:46:06 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534766, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:06.677062 27998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "cba0a2a94e6545569570dba5790cc58b"
format_stamp: "Formatted at 2026-09-27 18:46:06 on dist-test-slave-vs05"
server_key: "cae76d7cd9ac5fbbc230a2d69d59a16f"
server_key_iv: "2cc91e9ed1a586fc3f892bce18f61448"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:06.677521 27998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "cba0a2a94e6545569570dba5790cc58b"
format_stamp: "Formatted at 2026-09-27 18:46:06 on dist-test-slave-vs05"
server_key: "cae76d7cd9ac5fbbc230a2d69d59a16f"
server_key_iv: "2cc91e9ed1a586fc3f892bce18f61448"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:06.679150 27998 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260927 18:46:06.690889 28013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:06.691084 27998 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:06.691193 27998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "cba0a2a94e6545569570dba5790cc58b"
format_stamp: "Formatted at 2026-09-27 18:46:06 on dist-test-slave-vs05"
server_key: "cae76d7cd9ac5fbbc230a2d69d59a16f"
server_key_iv: "2cc91e9ed1a586fc3f892bce18f61448"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:06.691315 27998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:06.723021 27998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:06.725196 27998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:06.725373 27998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:06.730265 27998 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46827
I20260927 18:46:06.730304 28065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46827 every 8 connection(s)
I20260927 18:46:06.730661 27998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:06.731446 28066 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:06.733523 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 27998
I20260927 18:46:06.733642 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:46:06.733808 15803 external_mini_cluster.cc:1428] Setting key e0cd4756f3867591e81a88fcb7738b45
I20260927 18:46:06.734552 28066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Bootstrap starting.
I20260927 18:46:06.735297 28066 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:06.735653 28066 log.cc:824] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:06.736369 28066 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: No bootstrap required, opened a new log
I20260927 18:46:06.738574 28066 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:06.738731 28066 raft_consensus.cc:374] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:06.738791 28066 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba0a2a94e6545569570dba5790cc58b, State: Initialized, Role: FOLLOWER
I20260927 18:46:06.738952 28066 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:06.739048 28066 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:06.739101 28066 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:06.739190 28066 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:06.739936 28066 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:06.740051 28066 leader_election.cc:302] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba0a2a94e6545569570dba5790cc58b; no voters:
I20260927 18:46:06.740258 28066 leader_election.cc:288] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:46:06.740309 28071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:06.740564 28071 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 LEADER]: Becoming Leader. State: Replica: cba0a2a94e6545569570dba5790cc58b, State: Running, Role: LEADER
I20260927 18:46:06.740645 28066 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:06.740720 28071 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:06.741087 28071 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cba0a2a94e6545569570dba5790cc58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } } }
I20260927 18:46:06.741214 28071 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:06.741372 28071 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba0a2a94e6545569570dba5790cc58b. Latest consensus state: current_term: 1 leader_uuid: "cba0a2a94e6545569570dba5790cc58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } } }
I20260927 18:46:06.741462 28071 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: This master's current role is: LEADER
Sep 27 18:46:06 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534766, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:06.742220 28066 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:46:06.744601 28079 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:06.745692 28079 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:06.750605 28069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:06.734295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56390 (local address 127.15.110.254:46827)
0927 18:46:06.734473 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:06.734483 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:06.734515 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:46:06.735019 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:06.735027 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:06.735076 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:06.735254 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:06.735975 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:06.737326 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:06.738417 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:06.739033 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:06.739138 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:06.742260 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:06.742264 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:06.742265 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:06.742276 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:06.748248 (+ 5972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:06.748763 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:06.748767 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:06.748767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:06.748892 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:06.749131 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:06.749132 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:06.749132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:06.749490 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:06.749648 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:06.750152 (+ 504us) server_negotiation.cc:299] Negotiation successful
0927 18:46:06.750336 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":67,"threads_started":1}
I20260927 18:46:06.759295 28079 catalog_manager.cc:1347] Generated new cluster ID: efd67b218f934eaebbea6f3f12376ad2
I20260927 18:46:06.759356 28079 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:06.772296 28079 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:46:06.773096 28079 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:06.776440 28079 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Generated new TSK 0
I20260927 18:46:06.776715 28079 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:46:07.475513 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:07.581441 28111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:07.581799 28111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:07.581871 28111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:07.583686 28111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:07.583751 28111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:07.583796 28111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:07.585960 28111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:07.586197 28111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:07.586459 28111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:07.592729 28116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:07.592892 28111 server_base.cc:1034] running on GCE node
W20260927 18:46:07.592716 28117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:07.592710 28119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:07.593187 28111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:07.593433 28111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:07.594595 28111 hybrid_clock.cc:648] HybridClock initialized: now 1790534767594583 us; error 31 us; skew 500 ppm
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:07.597045 28111 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:07.597538 28111 webserver.cc:460] Webserver started at http://127.15.110.193:40109/ using document root <none> and password file <none>
I20260927 18:46:07.597787 28111 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:07.597831 28111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:07.597957 28111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:07.613978 28111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "dca8f971775d45cdbf64298ff9f83c07"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "d13c7c5b89207259509846963a68906b"
server_key_iv: "6f4c8ab924e8732eca2b2b6c1522e7f2"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.614477 28111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "dca8f971775d45cdbf64298ff9f83c07"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "d13c7c5b89207259509846963a68906b"
server_key_iv: "6f4c8ab924e8732eca2b2b6c1522e7f2"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.620105 28111 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260927 18:46:07.639307 28128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.639516 28111 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260927 18:46:07.639603 28111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "dca8f971775d45cdbf64298ff9f83c07"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "d13c7c5b89207259509846963a68906b"
server_key_iv: "6f4c8ab924e8732eca2b2b6c1522e7f2"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.639704 28111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:07.658674 28111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:07.661294 28111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:07.661523 28111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:07.661829 28111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:07.662268 28111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:07.662355 28111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.662451 28111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:07.662520 28111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.669685 28111 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39403
I20260927 18:46:07.670344 28111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:07.670401 28241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39403 every 8 connection(s)
I20260927 18:46:07.672196 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28111
I20260927 18:46:07.672294 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:46:07.672485 15803 external_mini_cluster.cc:1428] Setting key fb165671a30a58737ab26cbc1042ba41
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:46:07.686235 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:07.671586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:53099 (local address 127.15.110.254:46827)
0927 18:46:07.675478 (+ 3892us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:07.675481 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:07.675491 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:46:07.675521 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:07.675522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:07.675532 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:07.675643 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:07.676537 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.677210 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.678821 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.679232 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.679304 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:07.682297 (+ 2993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:07.682299 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:07.682299 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:07.682307 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:07.684216 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:07.684739 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:07.684740 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:07.684741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:07.684818 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:07.685147 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:07.685148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:07.685149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:07.685290 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:07.685343 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:07.686030 (+ 687us) server_negotiation.cc:299] Negotiation successful
0927 18:46:07.686123 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3843,"thread_start_us":82,"threads_started":1}
I20260927 18:46:07.686415 28244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:07.671314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.193:53099)
0927 18:46:07.671484 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:07.671529 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:07.672935 (+ 1406us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:07.675765 (+ 2830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:07.675771 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:07.676069 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:07.676402 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:07.676410 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.677322 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.677323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:07.678713 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:07.678717 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.679391 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.679392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:07.679548 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:07.680343 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:07.680355 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:07.682184 (+ 1829us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:07.684298 (+ 2114us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:07.684305 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:07.684307 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:07.684642 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:07.684896 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:07.684897 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:07.684898 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:07.685066 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:07.685683 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:07.685691 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:07.685860 (+ 169us) client_negotiation.cc:769] Sending connection context
0927 18:46:07.685986 (+ 126us) client_negotiation.cc:240] Negotiation successful
0927 18:46:07.686152 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260927 18:46:07.687070 28242 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:07.687211 28242 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:07.687414 28242 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
I20260927 18:46:07.687965 28030 ts_manager.cc:194] Registered new tserver with Master: dca8f971775d45cdbf64298ff9f83c07 (127.15.110.193:39403)
I20260927 18:46:07.688855 28030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:53099
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:46:07.705583 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:07.804569 28248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:07.804795 28248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:07.804836 28248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:07.806310 28248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:07.806371 28248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:07.806425 28248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:07.808071 28248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:07.808331 28248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:07.808591 28248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:07.814070 28258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:07.814429 28248 server_base.cc:1034] running on GCE node
W20260927 18:46:07.814160 28255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:07.814070 28256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:07.814801 28248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:07.815092 28248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:07.822485 28248 hybrid_clock.cc:648] HybridClock initialized: now 1790534767822445 us; error 42 us; skew 500 ppm
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:07.825548 28248 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:07.826128 28248 webserver.cc:460] Webserver started at http://127.15.110.194:33831/ using document root <none> and password file <none>
I20260927 18:46:07.826386 28248 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:07.826442 28248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:07.826573 28248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:07.843768 28248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "b9f1238adade4757beabdc03a00932d9"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "9b0577d13e9bc0b0421f0e247f911518"
server_key_iv: "71616b924fdbc205280cd88adfd426e3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.844233 28248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "b9f1238adade4757beabdc03a00932d9"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "9b0577d13e9bc0b0421f0e247f911518"
server_key_iv: "71616b924fdbc205280cd88adfd426e3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.846927 28248 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:07.869110 28265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.869402 28248 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.001s sys 0.000s
I20260927 18:46:07.869493 28248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "b9f1238adade4757beabdc03a00932d9"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "9b0577d13e9bc0b0421f0e247f911518"
server_key_iv: "71616b924fdbc205280cd88adfd426e3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:07.869594 28248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:07.907732 28248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:07.910343 28248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:07.910563 28248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:07.910915 28248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:07.911401 28248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:07.911491 28248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.911566 28248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:07.911623 28248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:07.917922 28248 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:42235
I20260927 18:46:07.918473 28248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:46:07.921896 28379 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:42235 every 8 connection(s)
Sep 27 18:46:07 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534767, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:07.929193 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28248
I20260927 18:46:07.929277 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:46:07.929471 15803 external_mini_cluster.cc:1428] Setting key b12f5dfb14b1ea9a6835240e55bb3f32
I20260927 18:46:07.935774 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:07.921558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:51669 (local address 127.15.110.254:46827)
0927 18:46:07.921614 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:07.921617 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:07.922384 (+ 767us) server_negotiation.cc:407] Connection header received
0927 18:46:07.923226 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:07.923227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:07.923243 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:07.923275 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:07.924098 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.924829 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.926368 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.926667 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.926717 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:07.932342 (+ 5625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:07.932344 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:07.932344 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:07.932352 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:07.934098 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:07.934588 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:07.934589 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:07.934590 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:07.934642 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:07.934893 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:07.934894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:07.934894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:07.935034 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:07.935076 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:07.935586 (+ 510us) server_negotiation.cc:299] Negotiation successful
0927 18:46:07.935684 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260927 18:46:07.936643 28382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:07.921283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.194:51669)
0927 18:46:07.922253 (+ 970us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:07.922299 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:07.923071 (+ 772us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:07.923467 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:07.923472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:07.923707 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:07.923988 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:07.923995 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.924945 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.924958 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:07.926224 (+ 1266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:07.926227 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:07.926822 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:07.926822 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:07.926975 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:07.927628 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:07.927638 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:07.932234 (+ 4596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:07.934195 (+ 1961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:07.934201 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:07.934204 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:07.934504 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:07.934708 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:07.934708 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:07.934709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:07.934826 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:07.935126 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:07.935133 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:07.935410 (+ 277us) client_negotiation.cc:769] Sending connection context
0927 18:46:07.936155 (+ 745us) client_negotiation.cc:240] Negotiation successful
0927 18:46:07.936331 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"thread_start_us":80,"threads_started":1}
I20260927 18:46:07.937305 28380 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:07.937400 28380 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:07.937585 28380 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:07.938015 28030 ts_manager.cc:194] Registered new tserver with Master: b9f1238adade4757beabdc03a00932d9 (127.15.110.194:42235)
I20260927 18:46:07.938488 28030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:51669
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:46:07.971280 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:08.080427 28385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:08.080809 28385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:08.080909 28385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:08.082966 28385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:08.083102 28385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:08.083225 28385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:08.085680 28385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:08.086333 28385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:08.086701 28385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:08.093886 28390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.093886 28391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.094558 28385 server_base.cc:1034] running on GCE node
W20260927 18:46:08.094712 28393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.094947 28385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:08.095293 28385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:08.098580 28385 hybrid_clock.cc:648] HybridClock initialized: now 1790534768098549 us; error 41 us; skew 500 ppm
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:08.101397 28385 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:08.101974 28385 webserver.cc:460] Webserver started at http://127.15.110.195:40189/ using document root <none> and password file <none>
I20260927 18:46:08.102227 28385 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:08.102301 28385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:08.102442 28385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:08.115942 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "36e0c4837c1941b2ab2374c96d0dd3bc"
format_stamp: "Formatted at 2026-09-27 18:46:08 on dist-test-slave-vs05"
server_key: "0539c02d8245e5d63da7cc44869a92c7"
server_key_iv: "d1e7bfe6cedd16982970baade82433a3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.120306 28385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "36e0c4837c1941b2ab2374c96d0dd3bc"
format_stamp: "Formatted at 2026-09-27 18:46:08 on dist-test-slave-vs05"
server_key: "0539c02d8245e5d63da7cc44869a92c7"
server_key_iv: "d1e7bfe6cedd16982970baade82433a3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.123960 28385 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:46:08.139374 28400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.139606 28385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:46:08.139724 28385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "36e0c4837c1941b2ab2374c96d0dd3bc"
format_stamp: "Formatted at 2026-09-27 18:46:08 on dist-test-slave-vs05"
server_key: "0539c02d8245e5d63da7cc44869a92c7"
server_key_iv: "d1e7bfe6cedd16982970baade82433a3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.139848 28385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:08.165777 28385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:08.170246 28385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:08.170459 28385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:08.170737 28385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:08.171139 28385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:08.171240 28385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.171304 28385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:08.171350 28385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.178040 28385 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:38551
I20260927 18:46:08.178557 28385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:08.179749 28513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:38551 every 8 connection(s)
I20260927 18:46:08.180001 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28385
I20260927 18:46:08.180091 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:46:08.180269 15803 external_mini_cluster.cc:1428] Setting key 2f13ea07a86fcffc178de66eacb0b8ed
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:08.191345 28245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.180745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42227 (local address 127.15.110.254:46827)
0927 18:46:08.180797 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:08.180799 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:08.180811 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:46:08.181444 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:08.181445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:08.181457 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:08.181507 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:08.182073 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.182635 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.183770 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.183982 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.184076 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.187464 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:08.187466 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:08.187466 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:08.187474 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:08.189533 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.190026 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.190028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.190028 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.190100 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.190449 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.190450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.190451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.190603 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:08.190661 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:08.191098 (+ 437us) server_negotiation.cc:299] Negotiation successful
0927 18:46:08.191238 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:46:08.192160 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.180439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.195:42227)
0927 18:46:08.180629 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:08.180684 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:08.181325 (+ 641us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:08.181587 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:08.181591 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:08.181781 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:08.181970 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.181974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.182731 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.182732 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.183664 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.183666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.184094 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.184095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.184211 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.185470 (+ 1259us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:08.185482 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:08.187336 (+ 1854us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:08.189629 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.189636 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.189639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.189930 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.190195 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.190196 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.190197 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.190348 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.190716 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:08.190722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:08.190914 (+ 192us) client_negotiation.cc:769] Sending connection context
0927 18:46:08.191706 (+ 792us) client_negotiation.cc:240] Negotiation successful
0927 18:46:08.191904 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":94,"threads_started":1}
I20260927 18:46:08.192930 28514 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:08.193032 28514 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.193233 28514 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:08.193769 28030 ts_manager.cc:194] Registered new tserver with Master: 36e0c4837c1941b2ab2374c96d0dd3bc (127.15.110.195:38551)
I20260927 18:46:08.194149 28030 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:42227
I20260927 18:46:08.202828 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:08.203788 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28111
I20260927 18:46:08.210815 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28248
I20260927 18:46:08.217118 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28385
I20260927 18:46:08.226109 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 27998
I20260927 18:46:08.236543 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46827
--webserver_interface=127.15.110.254
--webserver_port=41255
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46827
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:08.325678 28517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:08.325975 28517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:08.326015 28517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:08.327381 28517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:08.327453 28517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:08.327481 28517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:08.327514 28517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:08.327545 28517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:08.329284 28517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46827
--ranger_config_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46827
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=41255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:08.329588 28517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:08.329836 28517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:46:08.335567 28517 server_base.cc:1034] running on GCE node
W20260927 18:46:08.335567 28525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.335520 28523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.335520 28522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.336135 28517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:08.336483 28517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:08.337669 28517 hybrid_clock.cc:648] HybridClock initialized: now 1790534768337658 us; error 67 us; skew 500 ppm
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:08.340312 28517 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:08.340761 28517 webserver.cc:460] Webserver started at http://127.15.110.254:41255/ using document root <none> and password file <none>
I20260927 18:46:08.340986 28517 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:08.341063 28517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:08.395045 28517 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:08.396037 28533 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.396253 28517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:08.396361 28517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "cba0a2a94e6545569570dba5790cc58b"
format_stamp: "Formatted at 2026-09-27 18:46:06 on dist-test-slave-vs05"
server_key: "cae76d7cd9ac5fbbc230a2d69d59a16f"
server_key_iv: "2cc91e9ed1a586fc3f892bce18f61448"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.396831 28517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:08.407615 28517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:08.409176 28517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:08.409291 28517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:08.413704 28517 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46827
I20260927 18:46:08.413748 28585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46827 every 8 connection(s)
I20260927 18:46:08.414108 28517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:08.415311 28586 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:46:08.416008 28586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Bootstrap starting.
I20260927 18:46:08.422618 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28517
I20260927 18:46:08.422966 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:39403
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:46827
--webserver_port=40109
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:08.424070 28586 log.cc:824] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:08.426163 28586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:08.426491 28586 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Bootstrap complete.
I20260927 18:46:08.429203 28586 raft_consensus.cc:348] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:08.429479 28586 raft_consensus.cc:729] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: cba0a2a94e6545569570dba5790cc58b, State: Initialized, Role: FOLLOWER
I20260927 18:46:08.429705 28586 consensus_queue.cc:260] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:08.429814 28586 raft_consensus.cc:388] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:08.429883 28586 raft_consensus.cc:482] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:08.429939 28586 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:46:08.431025 28586 raft_consensus.cc:504] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:08.431159 28586 leader_election.cc:302] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cba0a2a94e6545569570dba5790cc58b; no voters:
I20260927 18:46:08.431409 28586 leader_election.cc:288] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:46:08.431444 28590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:46:08.431612 28590 raft_consensus.cc:686] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [term 2 LEADER]: Becoming Leader. State: Replica: cba0a2a94e6545569570dba5790cc58b, State: Running, Role: LEADER
I20260927 18:46:08.431718 28586 sys_catalog.cc:564] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:08.431784 28590 consensus_queue.cc:237] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } }
I20260927 18:46:08.432154 28591 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "cba0a2a94e6545569570dba5790cc58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } } }
I20260927 18:46:08.432248 28591 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:08.432549 28592 sys_catalog.cc:455] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: SysCatalogTable state changed. Reason: New leader cba0a2a94e6545569570dba5790cc58b. Latest consensus state: current_term: 2 leader_uuid: "cba0a2a94e6545569570dba5790cc58b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cba0a2a94e6545569570dba5790cc58b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46827 } } }
I20260927 18:46:08.432631 28592 sys_catalog.cc:458] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:08.432871 28596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:08.433341 28596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:08.433501 28596 catalog_manager.cc:1259] Loaded cluster ID: efd67b218f934eaebbea6f3f12376ad2
I20260927 18:46:08.433573 28596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:08.433689 28586 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260927 18:46:08.437216 28596 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:08.437829 28596 catalog_manager.cc:5879] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Loaded TSK: 0
I20260927 18:46:08.438123 28596 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:46:08.530855 28588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:08.531081 28588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:08.531117 28588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:08.532725 28588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:08.532791 28588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:08.532835 28588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:08.534430 28588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:39403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=40109
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:08.534693 28588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:08.534931 28588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:08.541229 28627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.541448 28624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.541203 28625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.544867 28588 server_base.cc:1034] running on GCE node
I20260927 18:46:08.545125 28588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:08.545409 28588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:08.546597 28588 hybrid_clock.cc:648] HybridClock initialized: now 1790534768546577 us; error 37 us; skew 500 ppm
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:08.549505 28588 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:08.550038 28588 webserver.cc:460] Webserver started at http://127.15.110.193:40109/ using document root <none> and password file <none>
I20260927 18:46:08.550299 28588 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:08.550395 28588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:08.579969 28588 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260927 18:46:08.580994 28635 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.581231 28588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:08.581334 28588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "dca8f971775d45cdbf64298ff9f83c07"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "d13c7c5b89207259509846963a68906b"
server_key_iv: "6f4c8ab924e8732eca2b2b6c1522e7f2"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.581785 28588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:08.597136 28588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:08.599601 28588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:08.599768 28588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:08.600077 28588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:08.600481 28588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:08.600534 28588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.600602 28588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:08.600646 28588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.608042 28588 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:39403
I20260927 18:46:08.608443 28588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:08.609054 28748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:39403 every 8 connection(s)
I20260927 18:46:08.611871 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28588
I20260927 18:46:08.612157 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:42235
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:46827
--webserver_port=33831
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:08.622377 28751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.609317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.193:38961)
0927 18:46:08.609477 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:08.609537 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:08.610198 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:08.610802 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:08.610806 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:08.611046 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:08.611336 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.611342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.613102 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.613103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.614386 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.614389 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.615096 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.615097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.615264 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.615901 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:08.615911 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:08.617676 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:08.620269 (+ 2593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.620277 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.620279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.620610 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.620968 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.620969 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.620969 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.621114 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.621654 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:08.621661 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:08.621826 (+ 165us) client_negotiation.cc:769] Sending connection context
0927 18:46:08.621965 (+ 139us) client_negotiation.cc:240] Negotiation successful
0927 18:46:08.622134 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260927 18:46:08.622468 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.609697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:38961 (local address 127.15.110.254:46827)
0927 18:46:08.609956 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:08.609964 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:08.609998 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:46:08.610527 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:08.610534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:08.610584 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:08.610709 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:08.611474 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.612971 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.614489 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.614921 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.615093 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.617818 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:08.617823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:08.617824 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:08.617839 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:08.620153 (+ 2314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.620734 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.620738 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.620739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.620876 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.621198 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.621199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.621199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.621556 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:08.621670 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:08.622053 (+ 383us) server_negotiation.cc:299] Negotiation successful
0927 18:46:08.622197 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":101,"threads_started":1}
I20260927 18:46:08.624148 28749 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:08.624266 28749 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.624557 28749 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:08.625488 28749 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:46827 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:46:08.625591 28749 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.625674 28749 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:08.625967 28749 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.626058 28749 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:08.626314 28749 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:46:08.718592 28753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:08.718943 28753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:08.719027 28753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:08.720793 28753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:08.720896 28753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:08.720974 28753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:08.722586 28753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:42235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=33831
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:08.722888 28753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:08.723145 28753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:08.729358 28759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.729709 28761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.730024 28758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.730341 28753 server_base.cc:1034] running on GCE node
I20260927 18:46:08.730530 28753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:08.730782 28753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:08.731945 28753 hybrid_clock.cc:648] HybridClock initialized: now 1790534768731882 us; error 74 us; skew 500 ppm
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:08.735522 28753 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:08.736060 28753 webserver.cc:460] Webserver started at http://127.15.110.194:33831/ using document root <none> and password file <none>
I20260927 18:46:08.736334 28753 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:08.736406 28753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:08.758425 28753 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260927 18:46:08.761934 28769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.762207 28753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260927 18:46:08.762339 28753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "b9f1238adade4757beabdc03a00932d9"
format_stamp: "Formatted at 2026-09-27 18:46:07 on dist-test-slave-vs05"
server_key: "9b0577d13e9bc0b0421f0e247f911518"
server_key_iv: "71616b924fdbc205280cd88adfd426e3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.762899 28753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:08.783149 28753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:08.785964 28753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:08.786193 28753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:08.786526 28753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:08.786942 28753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:08.787038 28753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.787109 28753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:08.787166 28753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.793821 28753 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:42235
I20260927 18:46:08.794395 28753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:46:08.796130 28882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:42235 every 8 connection(s)
I20260927 18:46:08.802198 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28753
I20260927 18:46:08.802513 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.15.110.212:33753/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:38551
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:46827
--webserver_port=40189
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:08.808101 28885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.796850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.194:38835)
0927 18:46:08.797272 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:08.797323 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:08.798000 (+ 677us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:08.798395 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:08.798400 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:08.798652 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:08.798945 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.798952 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.799966 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.799967 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.801318 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:08.801321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.801867 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.801868 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:08.802056 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.802691 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:08.802703 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:08.804394 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:08.806361 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.806368 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.806371 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.806681 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.806917 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:08.806918 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:08.806918 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:08.807052 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:08.807386 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:08.807392 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:08.807571 (+ 179us) client_negotiation.cc:769] Sending connection context
0927 18:46:08.807689 (+ 118us) client_negotiation.cc:240] Negotiation successful
0927 18:46:08.807853 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":84,"threads_started":1}
I20260927 18:46:08.808621 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:08.797162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:38835 (local address 127.15.110.254:46827)
0927 18:46:08.797224 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:08.797226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:08.797411 (+ 185us) server_negotiation.cc:407] Connection header received
0927 18:46:08.798138 (+ 727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:08.798139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:08.798155 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:08.798189 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:08.799092 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.799872 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.801419 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:08.801733 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:08.801812 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:08.804535 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:08.804537 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:08.804537 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:08.804545 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:08.806264 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.806768 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.806769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.806770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.806828 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:08.807127 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:08.807128 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:08.807128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:08.807289 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:08.807352 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:08.808430 (+ 1078us) server_negotiation.cc:299] Negotiation successful
0927 18:46:08.808530 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:46:08.809593 28883 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:08.809702 28883 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.809914 28883 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:08.810629 28883 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:46827 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:46:08.810719 28883 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.810794 28883 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:08.811138 28883 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:08.811242 28883 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:08.811584 28883 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:46:08.930596 28886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:08.930965 28886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:08.931082 28886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:08.933439 28886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:08.933547 28886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:08.933643 28886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:08.936034 28886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.15.110.212:33753/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:38551
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=40189
--tserver_master_addrs=127.15.110.254:46827
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:08.936514 28886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:08.936862 28886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:08.943998 28894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.944028 28891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:08.944053 28892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:08.945011 28886 server_base.cc:1034] running on GCE node
I20260927 18:46:08.945225 28886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:08.945482 28886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:08.946676 28886 hybrid_clock.cc:648] HybridClock initialized: now 1790534768946638 us; error 48 us; skew 500 ppm
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:08.950179 28886 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:08.950775 28886 webserver.cc:460] Webserver started at http://127.15.110.195:40189/ using document root <none> and password file <none>
I20260927 18:46:08.951043 28886 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:08.951112 28886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 27 18:46:08 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:08.983083 28886 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.000s
I20260927 18:46:08.984310 28903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:08.984536 28886 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:08.984634 28886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "36e0c4837c1941b2ab2374c96d0dd3bc"
format_stamp: "Formatted at 2026-09-27 18:46:08 on dist-test-slave-vs05"
server_key: "0539c02d8245e5d63da7cc44869a92c7"
server_key_iv: "d1e7bfe6cedd16982970baade82433a3"
server_key_version: "kuduclusterkey@0"
I20260927 18:46:08.985148 28886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:09.001405 28886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:09.004083 28886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:09.004333 28886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:09.004644 28886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:09.005103 28886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:09.005198 28886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:09.005273 28886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:09.005333 28886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:09.013263 28886 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:38551
I20260927 18:46:09.013808 28886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:09.014242 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 28886
Sep 27 18:46:09 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534768, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:09.026054 28752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:09.014759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56394 (local address 127.15.110.254:46827)
0927 18:46:09.014813 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:09.014815 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:09.016835 (+ 2020us) server_negotiation.cc:407] Connection header received
0927 18:46:09.017014 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:09.017015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:09.017027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:09.017062 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:09.017929 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.018672 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.019762 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.020054 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.020091 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:09.021666 (+ 1575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:09.021667 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:09.021667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:09.021674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:09.023820 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:09.025064 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:09.025065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:09.025066 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:09.025125 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:09.025345 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:09.025346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:09.025346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:09.025476 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:09.025618 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:09.025849 (+ 231us) server_negotiation.cc:299] Negotiation successful
0927 18:46:09.025938 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:46:09.015532 29016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:38551 every 8 connection(s)
I20260927 18:46:09.032223 29021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:09.016570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:45815 (local address 127.15.110.254:46827)
0927 18:46:09.016885 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:09.016888 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:09.018566 (+ 1678us) server_negotiation.cc:407] Connection header received
0927 18:46:09.018587 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:09.018588 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:09.018599 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:09.018656 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:09.019388 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.020102 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.021662 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.021983 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.022043 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:09.028066 (+ 6023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:09.028067 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:09.028067 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:09.028074 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:09.029703 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:09.030926 (+ 1223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:09.030928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:09.030928 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:09.030994 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:09.031375 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:09.031376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:09.031376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:09.031531 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:09.031581 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:09.032012 (+ 431us) server_negotiation.cc:299] Negotiation successful
0927 18:46:09.032101 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":56,"threads_started":1}
I20260927 18:46:09.032783 29020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:09.016277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46827 (local address 127.15.110.195:45815)
0927 18:46:09.017755 (+ 1478us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:09.017809 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:09.018418 (+ 609us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:09.018773 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:09.018777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:09.018992 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:09.019285 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:09.019292 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.020199 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.020199 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:09.021579 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:09.021582 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:09.022135 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:09.022137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:09.022290 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:09.022852 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:09.022863 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:09.027921 (+ 5058us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:09.030481 (+ 2560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:09.030489 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:09.030491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:09.030823 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:09.031073 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:09.031074 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:09.031074 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:09.031248 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:09.031652 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:09.031659 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:09.031838 (+ 179us) client_negotiation.cc:769] Sending connection context
0927 18:46:09.032317 (+ 479us) client_negotiation.cc:240] Negotiation successful
0927 18:46:09.032517 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1427,"thread_start_us":93,"threads_started":1}
I20260927 18:46:09.033406 29017 heartbeater.cc:347] Connected to a master server at 127.15.110.254:46827
I20260927 18:46:09.033505 29017 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:09.033701 29017 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:09.034387 29017 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:46827 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260927 18:46:09.034452 29017 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:09.034513 29017 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:09.034835 29017 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:09.034905 29017 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
W20260927 18:46:09.035290 29017 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 27 18:46:09 dist-test-slave-vs05 krb5kdc[27554](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790534769, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:09.626494 28749 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:09.626647 28749 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:09.626969 28550 ts_manager.cc:194] Registered new tserver with Master: dca8f971775d45cdbf64298ff9f83c07 (127.15.110.193:39403)
I20260927 18:46:09.628027 28550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:38961
I20260927 18:46:09.811755 28883 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:09.811933 28883 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:09.812193 28550 ts_manager.cc:194] Registered new tserver with Master: b9f1238adade4757beabdc03a00932d9 (127.15.110.194:42235)
I20260927 18:46:09.812649 28550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:38835
Sep 27 18:46:10 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534769, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
I20260927 18:46:10.035475 29017 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:10.035619 29017 heartbeater.cc:511] Master 127.15.110.254:46827 requested a full tablet report, sending...
I20260927 18:46:10.035961 28550 ts_manager.cc:194] Registered new tserver with Master: 36e0c4837c1941b2ab2374c96d0dd3bc (127.15.110.195:38551)
I20260927 18:46:10.036487 28550 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:45815
I20260927 18:46:10.044628 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:10.054124 29038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.045604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56400 (local address 127.15.110.254:46827)
0927 18:46:10.045905 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.045907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.046114 (+ 207us) server_negotiation.cc:407] Connection header received
0927 18:46:10.046139 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.046140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.046150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.046205 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:10.046906 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.047669 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.048651 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.048943 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.049022 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.050566 (+ 1544us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:10.050567 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:10.050568 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:10.050575 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:10.052364 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:10.052829 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:10.052830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:10.052831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:10.053178 (+ 347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:10.053450 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:10.053451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:10.053452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:10.053605 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:10.053704 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.053891 (+ 187us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.053989 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":64,"threads_started":1}
I20260927 18:46:10.056294 28550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:46:10.058887 28550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:46:10.089819 29047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.078544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39403 (local address 127.0.0.1:52978)
0927 18:46:10.084254 (+ 5710us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.084264 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.084309 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.084660 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:10.084664 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:10.084666 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:10.084834 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.084838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.086549 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.086550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.087983 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.087987 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.089065 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.089067 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.089219 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.089240 (+ 21us) client_negotiation.cc:769] Sending connection context
0927 18:46:10.089312 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:46:10.089400 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5692,"thread_start_us":52,"threads_started":1}
I20260927 18:46:10.090554 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.079045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52978 (local address 127.15.110.193:39403)
0927 18:46:10.084416 (+ 5371us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.084420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.084430 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:46:10.084462 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.084464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.084480 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.084550 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:10.084934 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.086453 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.088092 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.088877 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.088954 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.089027 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.090374 (+ 1347us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.090474 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5337,"thread_start_us":48,"threads_started":1}
I20260927 18:46:10.091459 28683 tablet_service.cc:1467] Processing CreateTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 (DEFAULT_TABLE table=test-table [id=07fc87dfd0c34dc585161216b8e9258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:10.091861 28683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a0a4b650f24168b9b6832ad9a84d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:10.098065 29045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.077929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:38551 (local address 127.0.0.1:35148)
0927 18:46:10.083306 (+ 5377us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.083361 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.083443 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.091045 (+ 7602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:10.091046 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:10.091050 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:10.091097 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.091098 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.092425 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.092426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.093556 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.093559 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.097799 (+ 4240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.097800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.097866 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.097871 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:10.097913 (+ 42us) client_negotiation.cc:240] Negotiation successful
0927 18:46:10.097961 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5335,"thread_start_us":84,"threads_started":1}
I20260927 18:46:10.098513 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.078199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35148 (local address 127.15.110.195:38551)
0927 18:46:10.083943 (+ 5744us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.083948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.083959 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:10.084007 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.084009 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.084024 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.084106 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:10.091218 (+ 7112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.092321 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.096801 (+ 4480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.097608 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.097696 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.097761 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.098377 (+ 616us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.098431 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5706,"thread_start_us":46,"threads_started":1}
I20260927 18:46:10.099833 28951 tablet_service.cc:1467] Processing CreateTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 (DEFAULT_TABLE table=test-table [id=07fc87dfd0c34dc585161216b8e9258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:10.100242 28951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a0a4b650f24168b9b6832ad9a84d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:10.105934 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.078884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38882 (local address 127.15.110.194:42235)
0927 18:46:10.083637 (+ 4753us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.083643 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.083654 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:10.083695 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.083697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.083715 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.083795 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:10.094103 (+ 10308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.095195 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.104270 (+ 9075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.105458 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.105514 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.105579 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.105805 (+ 226us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.105850 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4712,"thread_start_us":66,"threads_started":1}
I20260927 18:46:10.106388 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.078398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42235 (local address 127.0.0.1:38882)
0927 18:46:10.083565 (+ 5167us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.083573 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.083604 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.093948 (+ 10344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:10.093949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:10.093951 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:10.094005 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.094006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.095300 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.095302 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.096675 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.096678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.105642 (+ 8964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.105642 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.105718 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.105722 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:46:10.106258 (+ 536us) client_negotiation.cc:240] Negotiation successful
0927 18:46:10.106315 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5144,"thread_start_us":60,"threads_started":1}
I20260927 18:46:10.107259 28798 tablet_service.cc:1467] Processing CreateTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 (DEFAULT_TABLE table=test-table [id=07fc87dfd0c34dc585161216b8e9258c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:10.107614 28798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c2a0a4b650f24168b9b6832ad9a84d04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:10.110158 29052 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Bootstrap starting.
I20260927 18:46:10.110878 29052 tablet_bootstrap.cc:654] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:10.111253 29052 log.cc:824] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:10.111341 29051 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Bootstrap starting.
I20260927 18:46:10.111974 29051 tablet_bootstrap.cc:654] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:10.112287 29051 log.cc:824] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:10.113090 29051 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: No bootstrap required, opened a new log
I20260927 18:46:10.113090 29052 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: No bootstrap required, opened a new log
I20260927 18:46:10.113157 29051 ts_tablet_manager.cc:1397] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:10.113157 29052 ts_tablet_manager.cc:1397] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260927 18:46:10.113968 29053 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Bootstrap starting.
I20260927 18:46:10.114559 29053 tablet_bootstrap.cc:654] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:10.114833 29053 log.cc:824] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:10.115592 29053 tablet_bootstrap.cc:492] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: No bootstrap required, opened a new log
I20260927 18:46:10.115480 29051 raft_consensus.cc:348] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.115649 29053 ts_tablet_manager.cc:1397] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:46:10.115648 29051 raft_consensus.cc:374] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:10.115682 29051 raft_consensus.cc:729] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dca8f971775d45cdbf64298ff9f83c07, State: Initialized, Role: FOLLOWER
I20260927 18:46:10.115751 29052 raft_consensus.cc:348] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.115924 29052 raft_consensus.cc:374] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:10.115985 29052 raft_consensus.cc:729] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36e0c4837c1941b2ab2374c96d0dd3bc, State: Initialized, Role: FOLLOWER
I20260927 18:46:10.116128 29052 consensus_queue.cc:260] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.116389 29052 ts_tablet_manager.cc:1428] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260927 18:46:10.116519 29017 heartbeater.cc:503] Master 127.15.110.254:46827 was elected leader, sending a full tablet report...
I20260927 18:46:10.117699 29053 raft_consensus.cc:348] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.117856 29053 raft_consensus.cc:374] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:10.117885 29053 raft_consensus.cc:729] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b9f1238adade4757beabdc03a00932d9, State: Initialized, Role: FOLLOWER
I20260927 18:46:10.117995 29053 consensus_queue.cc:260] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.118228 29053 ts_tablet_manager.cc:1428] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:46:10.118278 28883 heartbeater.cc:503] Master 127.15.110.254:46827 was elected leader, sending a full tablet report...
I20260927 18:46:10.123531 29051 consensus_queue.cc:260] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.123878 29051 ts_tablet_manager.cc:1428] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260927 18:46:10.124018 28749 heartbeater.cc:503] Master 127.15.110.254:46827 was elected leader, sending a full tablet report...
W20260927 18:46:10.266577 29018 tablet.cc:2369] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 27 18:46:10 dist-test-slave-vs05 krb5kdc[27554](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534769, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for HTTP/127.15.110.212@KRBTEST.COM
W20260927 18:46:10.297230 28884 tablet.cc:2369] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:10.314296 29057 raft_consensus.cc:482] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:10.314473 29057 raft_consensus.cc:504] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.314891 29057 leader_election.cc:288] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b9f1238adade4757beabdc03a00932d9 (127.15.110.194:42235), dca8f971775d45cdbf64298ff9f83c07 (127.15.110.193:39403)
I20260927 18:46:10.319585 29049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.315559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:36007 (local address 127.15.110.194:42235)
0927 18:46:10.315624 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.315626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.315676 (+ 50us) server_negotiation.cc:407] Connection header received
0927 18:46:10.315705 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.315706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.315717 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.315774 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:10.316003 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.316691 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.318099 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.318651 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.318706 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.318716 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.319458 (+ 742us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.319507 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260927 18:46:10.329276 29050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.315622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:34915 (local address 127.15.110.193:39403)
0927 18:46:10.315841 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.315843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.322024 (+ 6181us) server_negotiation.cc:407] Connection header received
0927 18:46:10.322606 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.322607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.322619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.322651 (+ 32us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:10.323399 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.324151 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.327900 (+ 3749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.328500 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.328554 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.328565 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.329160 (+ 595us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.329209 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260927 18:46:10.329282 29061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.315064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42235 (local address 127.15.110.195:36007)
0927 18:46:10.315552 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.315566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.315618 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.315802 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:10.315804 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:10.315806 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:10.315892 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.315895 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.316799 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.316800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.317992 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.317996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.319283 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.319284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.319369 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.319374 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:10.329104 (+ 9730us) client_negotiation.cc:240] Negotiation successful
0927 18:46:10.329169 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":131,"threads_started":1}
I20260927 18:46:10.329674 29062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.315314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39403 (local address 127.15.110.195:34915)
0927 18:46:10.321950 (+ 6636us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.321964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.322499 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.322722 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:10.322723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:10.322725 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:10.322837 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.322839 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.324260 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.324260 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.325453 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:10.325465 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.328634 (+ 3169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.328635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:10.328716 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.328723 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:46:10.329556 (+ 833us) client_negotiation.cc:240] Negotiation successful
0927 18:46:10.329600 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6612,"thread_start_us":62,"threads_started":1}
I20260927 18:46:10.330054 28833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a0a4b650f24168b9b6832ad9a84d04" candidate_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f1238adade4757beabdc03a00932d9" is_pre_election: true
I20260927 18:46:10.330148 28703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a0a4b650f24168b9b6832ad9a84d04" candidate_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca8f971775d45cdbf64298ff9f83c07" is_pre_election: true
I20260927 18:46:10.330253 28833 raft_consensus.cc:2454] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36e0c4837c1941b2ab2374c96d0dd3bc in term 0.
I20260927 18:46:10.330299 28703 raft_consensus.cc:2454] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 36e0c4837c1941b2ab2374c96d0dd3bc in term 0.
I20260927 18:46:10.330608 28905 leader_election.cc:302] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e0c4837c1941b2ab2374c96d0dd3bc, b9f1238adade4757beabdc03a00932d9; no voters:
I20260927 18:46:10.330834 29057 raft_consensus.cc:2781] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:10.330875 29057 raft_consensus.cc:482] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:10.330890 29057 raft_consensus.cc:3037] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:10.331857 29057 raft_consensus.cc:504] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.331998 29057 leader_election.cc:288] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [CANDIDATE]: Term 1 election: Requested vote from peers b9f1238adade4757beabdc03a00932d9 (127.15.110.194:42235), dca8f971775d45cdbf64298ff9f83c07 (127.15.110.193:39403)
I20260927 18:46:10.332605 28833 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a0a4b650f24168b9b6832ad9a84d04" candidate_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b9f1238adade4757beabdc03a00932d9"
I20260927 18:46:10.332693 28833 raft_consensus.cc:3037] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:10.333722 28833 raft_consensus.cc:2454] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36e0c4837c1941b2ab2374c96d0dd3bc in term 1.
I20260927 18:46:10.334188 28703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c2a0a4b650f24168b9b6832ad9a84d04" candidate_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dca8f971775d45cdbf64298ff9f83c07"
I20260927 18:46:10.334276 28703 raft_consensus.cc:3037] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:10.335335 28703 raft_consensus.cc:2454] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 36e0c4837c1941b2ab2374c96d0dd3bc in term 1.
I20260927 18:46:10.335659 28905 leader_election.cc:302] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36e0c4837c1941b2ab2374c96d0dd3bc, dca8f971775d45cdbf64298ff9f83c07; no voters:
I20260927 18:46:10.335862 29057 raft_consensus.cc:2781] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:10.336037 29057 raft_consensus.cc:686] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 1 LEADER]: Becoming Leader. State: Replica: 36e0c4837c1941b2ab2374c96d0dd3bc, State: Running, Role: LEADER
I20260927 18:46:10.336125 29057 consensus_queue.cc:237] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } }
I20260927 18:46:10.337055 28550 catalog_manager.cc:5495] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 36e0c4837c1941b2ab2374c96d0dd3bc (127.15.110.195). New cstate: current_term: 1 leader_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "36e0c4837c1941b2ab2374c96d0dd3bc" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 38551 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 } health_report { overall_health: UNKNOWN } } }
W20260927 18:46:10.361060 28750 tablet.cc:2369] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:10.365805 29046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.362431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39734 (local address 127.15.110.195:38551)
0927 18:46:10.362483 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:10.362486 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:10.362495 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:46:10.362554 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:10.362555 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:10.362579 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:10.362657 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:10.362917 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.363686 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.364743 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:10.365059 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:10.365134 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:10.365409 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:10.365550 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:10.365642 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:10.365697 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:46:10.365722 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:46:10.368350 28703 raft_consensus.cc:1261] T c2a0a4b650f24168b9b6832ad9a84d04 P dca8f971775d45cdbf64298ff9f83c07 [term 1 FOLLOWER]: Refusing update from remote peer 36e0c4837c1941b2ab2374c96d0dd3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:10.368794 29064 consensus_queue.cc:1035] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "dca8f971775d45cdbf64298ff9f83c07" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 39403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:10.370903 28833 raft_consensus.cc:1261] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 1 FOLLOWER]: Refusing update from remote peer 36e0c4837c1941b2ab2374c96d0dd3bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:10.371244 29064 consensus_queue.cc:1035] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "b9f1238adade4757beabdc03a00932d9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:10.374888 29067 mvcc.cc:205] Tried to move back new op lower bound from 7334030419425533952 to 7334030419297521664. Current Snapshot: MvccSnapshot[applied={T|T < 7334030419425533952}]
I20260927 18:46:10.375576 29070 mvcc.cc:205] Tried to move back new op lower bound from 7334030419425533952 to 7334030419297521664. Current Snapshot: MvccSnapshot[applied={T|T < 7334030419425533952}]
I20260927 18:46:10.377727 29068 mvcc.cc:205] Tried to move back new op lower bound from 7334030419425533952 to 7334030419297521664. Current Snapshot: MvccSnapshot[applied={T|T < 7334030419425533952}]
I20260927 18:46:10.380100 28550 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:10.380190 28550 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56400:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:10.380963 28550 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cba0a2a94e6545569570dba5790cc58b: Sending DeleteTablet for 3 replicas of tablet c2a0a4b650f24168b9b6832ad9a84d04
I20260927 18:46:10.381273 28951 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:10 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:35148
I20260927 18:46:10.381511 28683 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:10 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:52978
I20260927 18:46:10.381496 28798 tablet_service.cc:1514] Processing DeleteTablet for tablet c2a0a4b650f24168b9b6832ad9a84d04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:10 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:38882
I20260927 18:46:10.382072 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28588
I20260927 18:46:10.391465 29078 tablet_replica.cc:330] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: stopping tablet replica
I20260927 18:46:10.391634 29076 tablet_replica.cc:330] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: stopping tablet replica
I20260927 18:46:10.391834 29076 raft_consensus.cc:2229] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:46:10.392020 29078 raft_consensus.cc:2229] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:10.392230 29076 raft_consensus.cc:2258] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:10.392413 29078 raft_consensus.cc:2258] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260927 18:46:10.392954 28535 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:39403 (error 108)
I20260927 18:46:10.392976 29076 ts_tablet_manager.cc:1905] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:10.393637 29078 ts_tablet_manager.cc:1905] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:10.394938 29076 ts_tablet_manager.cc:1918] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:10.395079 29076 log.cc:1196] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/c2a0a4b650f24168b9b6832ad9a84d04
I20260927 18:46:10.395471 29076 ts_tablet_manager.cc:1939] T c2a0a4b650f24168b9b6832ad9a84d04 P 36e0c4837c1941b2ab2374c96d0dd3bc: Deleting consensus metadata
I20260927 18:46:10.395709 29078 ts_tablet_manager.cc:1918] T c2a0a4b650f24168b9b6832ad9a84d04 P b9f1238adade4757beabdc03a00932d9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:10.395934 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28753
I20260927 18:46:10.396631 28536 catalog_manager.cc:4841] TS 36e0c4837c1941b2ab2374c96d0dd3bc (127.15.110.195:38551): tablet c2a0a4b650f24168b9b6832ad9a84d04 (table test-table [id=07fc87dfd0c34dc585161216b8e9258c]) successfully deleted
I20260927 18:46:10.407617 29048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:10.407474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:39403 (local address 127.0.0.1:54892)
0927 18:46:10.407520 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.407564 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260927 18:46:10.407893 28535 catalog_manager.cc:4584] TS dca8f971775d45cdbf64298ff9f83c07 (127.15.110.193:39403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c2a0a4b650f24168b9b6832ad9a84d04: Network error: Client connection negotiation failed: client connection to 127.15.110.193:39403: connect: Connection refused (error 111)
W20260927 18:46:10.408412 29048 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:46:10.408065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42235 (local address 127.0.0.1:56030)
0927 18:46:10.408195 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:10.408205 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:10.408247 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:10.408373 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.194:42235: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":112}
I20260927 18:46:10.411520 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28886
I20260927 18:46:10.419765 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 28517
I20260927 18:46:10.427708 15803 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260927 18:46:10.772369 15803 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260927 18:46:10.772470 15803 mini_ranger.cc:66] Stopping Ranger...
I20260927 18:46:11.117548 15803 mini_ranger.cc:68] Stopped Ranger
2026-09-27 18:46:11.117 UTC [27584] LOG: received smart shutdown request
2026-09-27 18:46:11.122 UTC [27584] LOG: background worker "logical replication launcher" (PID 27593) exited with exit code 1
2026-09-27 18:46:11.122 UTC [27588] LOG: shutting down
2026-09-27 18:46:11.314 UTC [27584] LOG: database system is shut down
2026-09-27T18:46:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (48812 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:46:11 dist-test-slave-vs05 krb5kdc[29088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:46:11 dist-test-slave-vs05 krb5kdc[29088](info): set up 2 sockets
Sep 27 18:46:11 dist-test-slave-vs05 krb5kdc[29088](info): commencing operation
krb5kdc: starting...
W20260927 18:46:12.464085 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:46:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:46:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:46:12.581754 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42327
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:12.663998 29101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:12.664255 29101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:12.664294 29101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:12.665462 29101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:12.665539 29101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:12.665563 29101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:12.665583 29101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:12.665611 29101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:12.667119 29101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:12.667462 29101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:12.667696 29101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:12.672703 29109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:12.672709 29106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:12.673038 29101 server_base.cc:1034] running on GCE node
W20260927 18:46:12.672716 29107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:12.673377 29101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:12.673730 29101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:12.674938 29101 hybrid_clock.cc:648] HybridClock initialized: now 1790534772674907 us; error 41 us; skew 500 ppm
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:12.683666 29101 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:12.684209 29101 webserver.cc:460] Webserver started at http://127.15.110.254:46833/ using document root <none> and password file <none>
I20260927 18:46:12.684458 29101 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:12.684508 29101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:12.684615 29101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:12.685751 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "95c3413755e9ccf5ac212e687dea47bf"
server_key_iv: "31f7f9bb89bb64b2709467c9069cdbb4"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.686117 29101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "95c3413755e9ccf5ac212e687dea47bf"
server_key_iv: "31f7f9bb89bb64b2709467c9069cdbb4"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.687590 29101 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:46:12.688392 29116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:12.688587 29101 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:46:12.688664 29101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "95c3413755e9ccf5ac212e687dea47bf"
server_key_iv: "31f7f9bb89bb64b2709467c9069cdbb4"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.688721 29101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:12.707053 29101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:12.709112 29101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:12.709226 29101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:12.713129 29101 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:42327
I20260927 18:46:12.713186 29168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:42327 every 8 connection(s)
I20260927 18:46:12.713528 29101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:12.714470 29169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:12.716446 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:12.716890 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29101
I20260927 18:46:12.717039 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:46:12.717264 15803 external_mini_cluster.cc:1428] Setting key bfe96b1d7fc3e6df860b044257c06d95
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.253@KRBTEST.COM; defaulting to no policy
I20260927 18:46:12.727845 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.717201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42327 (local address 127.0.0.1:46818)
0927 18:46:12.717456 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.717504 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:12.718227 (+ 723us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:12.718521 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:12.718524 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:12.718908 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:12.719077 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.719081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.720636 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.720637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.721396 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.721398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.722143 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.722144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.722197 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.722763 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:12.722785 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:12.724433 (+ 1648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:12.726364 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.726370 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.726372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.726563 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.726849 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.726850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.726850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.726932 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.727327 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:12.727333 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:12.727389 (+ 56us) client_negotiation.cc:769] Sending connection context
0927 18:46:12.727475 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:46:12.727652 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"spinlock_wait_cycles":9088,"thread_start_us":87,"threads_started":1}
I20260927 18:46:12.727850 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.717210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46818 (local address 127.15.110.254:42327)
0927 18:46:12.717387 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:12.717396 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:12.717608 (+ 212us) server_negotiation.cc:407] Connection header received
0927 18:46:12.718370 (+ 762us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:12.718372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:12.718395 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:12.718472 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:12.719202 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.720541 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.721502 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.722051 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.722173 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.724536 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:12.724545 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:12.724546 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:12.724554 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:12.726291 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.726699 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.726702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.726702 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.726790 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.726994 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.726994 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.726994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.727267 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:12.727332 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:12.727557 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 18:46:12.727709 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":140,"thread_start_us":87,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.253@KRBTEST.COM" created.
I20260927 18:46:12.728684 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:12.729061 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.728924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42686)
0927 18:46:12.728977 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.729028 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 18:46:12.729216 29120 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
I20260927 18:46:12.729665 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.729554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42688)
0927 18:46:12.729606 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.729636 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:46:12.729775 29169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35657: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
I20260927 18:46:12.742401 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:35657
--webserver_interface=127.15.110.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:12.779011 29169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } attempt: 1
I20260927 18:46:12.779544 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.779376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42698)
0927 18:46:12.779453 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.779503 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 18:46:12.779947 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.779851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42712)
0927 18:46:12.779898 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.779927 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:46:12.780108 29169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35657: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
W20260927 18:46:12.824808 29176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:12.825045 29176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:12.825068 29176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:12.826246 29176 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:12.826292 29176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:12.826306 29176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:12.826319 29176 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:12.826331 29176 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:12.827905 29176 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.110.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:12.828220 29176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:12.828469 29176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:12.833396 29184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:12.833482 29181 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:12.833412 29182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:12.833441 29176 server_base.cc:1034] running on GCE node
I20260927 18:46:12.833779 29176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:12.834080 29176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:12.835258 29176 hybrid_clock.cc:648] HybridClock initialized: now 1790534772835236 us; error 36 us; skew 500 ppm
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:12.837687 29176 init.cc:377] Logged in from keytab as kudu/127.15.110.253@KRBTEST.COM (short username kudu)
I20260927 18:46:12.838135 29176 webserver.cc:460] Webserver started at http://127.15.110.253:46077/ using document root <none> and password file <none>
I20260927 18:46:12.838312 29176 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:12.838352 29176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:12.838438 29176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:12.839491 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data/instance:
uuid: "dd3ab5dcaef443fca9a773cadf6e700c"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "a9088cecd962351e59992ddb6c488c7e"
server_key_iv: "ed8ffbe09beed724e58efe8b24116ab5"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.839792 29176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal/instance:
uuid: "dd3ab5dcaef443fca9a773cadf6e700c"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "a9088cecd962351e59992ddb6c488c7e"
server_key_iv: "ed8ffbe09beed724e58efe8b24116ab5"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.841130 29176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:46:12.841812 29191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:12.842034 29176 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:12.842141 29176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "dd3ab5dcaef443fca9a773cadf6e700c"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "a9088cecd962351e59992ddb6c488c7e"
server_key_iv: "ed8ffbe09beed724e58efe8b24116ab5"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.842216 29176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:12.848273 29169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } attempt: 2
I20260927 18:46:12.848666 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.848528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42722)
0927 18:46:12.848589 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.848630 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:46:12.848984 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.848901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42728)
0927 18:46:12.848943 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.848967 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:46:12.849103 29169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35657: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35657: connect: Connection refused (error 111)
I20260927 18:46:12.854138 29176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:12.856092 29176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:12.856252 29176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:12.860481 29176 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.253:35657
I20260927 18:46:12.860502 29243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.253:35657 every 8 connection(s)
I20260927 18:46:12.860862 29176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
I20260927 18:46:12.861758 29244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:12.863360 29244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:12.867074 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29176
I20260927 18:46:12.867205 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-1/wal/instance
I20260927 18:46:12.867424 15803 external_mini_cluster.cc:1428] Setting key 8322a6c6f3481f3473b307f14662a654
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:12.872999 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.864010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42327 (local address 127.0.0.1:46822)
0927 18:46:12.864240 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.864287 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:12.864770 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:12.865035 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:12.865038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:12.865283 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:12.865479 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.865483 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.866240 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.866241 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.867027 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.867029 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.867585 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.867586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.867754 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.868249 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:12.868259 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:12.869833 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:12.871666 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.871672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.871673 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.871885 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.872097 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.872098 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.872098 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.872194 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.872508 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:12.872513 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:12.872627 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 18:46:12.872712 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:46:12.872829 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":99,"threads_started":1}
I20260927 18:46:12.872995 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.864097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46822 (local address 127.15.110.254:42327)
0927 18:46:12.864191 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:12.864194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:12.864383 (+ 189us) server_negotiation.cc:407] Connection header received
0927 18:46:12.864887 (+ 504us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:12.864888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:12.864904 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:12.864961 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:12.865586 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.866121 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.867138 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.867449 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.867540 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.869925 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:12.869926 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:12.869926 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:12.869934 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:12.871583 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.871965 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.871966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.871967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.872028 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.872265 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.872266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.872266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.872433 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:12.872511 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:12.872766 (+ 255us) server_negotiation.cc:299] Negotiation successful
0927 18:46:12.872861 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260927 18:46:12.874049 29244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.110.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.252@KRBTEST.COM" created.
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:46:12.882481 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.874354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42734)
0927 18:46:12.874411 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.874423 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:12.874468 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:12.874735 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:12.874736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:12.874869 (+ 133us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:12.874912 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.874913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.875904 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.875905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.876397 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.876399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.876896 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.876896 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.876963 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.877340 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:12.877344 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:12.879391 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:12.881451 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.881452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.881452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.881615 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.881836 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.881837 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.881837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.881890 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.882179 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:12.882180 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:12.882217 (+ 37us) client_negotiation.cc:769] Sending connection context
0927 18:46:12.882277 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:46:12.882373 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:46:12.882586 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.874453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42734 (local address 127.15.110.253:35657)
0927 18:46:12.874595 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:12.874599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:12.874616 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:46:12.874634 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:12.874635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:12.874652 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:12.874735 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:12.874989 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.875803 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.876502 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.876807 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.876898 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.879492 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:12.879501 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:12.879501 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:12.879508 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:12.881348 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.881709 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.881712 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.881713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.881768 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.881962 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.881963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.881963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.882106 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:12.882204 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:12.882315 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 18:46:12.882448 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260927 18:46:12.883162 29244 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:12.883565 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.883461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49130)
0927 18:46:12.883505 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.883545 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 18:46:12.883737 29192 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
I20260927 18:46:12.884186 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.884098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49144)
0927 18:46:12.884146 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.884169 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:46:12.884315 29244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36577: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
I20260927 18:46:12.895738 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36577
--webserver_interface=127.15.110.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:12.933526 29244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } attempt: 1
I20260927 18:46:12.934022 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.933883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49148)
0927 18:46:12.933956 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.933992 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:46:12.934360 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.934262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49152)
0927 18:46:12.934310 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.934334 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:46:12.934463 29244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36577: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
I20260927 18:46:12.940237 29169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } attempt: 3
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:46:12.947170 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.940517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42742)
0927 18:46:12.940577 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.940588 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:12.940654 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:12.940796 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:12.940797 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:12.940960 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:12.941016 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.941018 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.941703 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.941705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.942217 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:12.942219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.942603 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.942604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:12.942686 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.943248 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:12.943253 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:12.944440 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:12.946142 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.946143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.946143 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.946313 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.946526 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:12.946527 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:12.946527 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:12.946588 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:12.946832 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:12.946833 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:12.946881 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 18:46:12.946975 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 18:46:12.947074 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:46:12.947168 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.940605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.15.110.253:35657)
0927 18:46:12.940672 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:12.940674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:12.940685 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:12.940711 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:12.940712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:12.940721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:12.940790 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:12.941107 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.941612 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.942313 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:12.942524 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:12.942606 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:12.944528 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:12.944530 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:12.944530 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:12.944538 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:12.946047 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.946396 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.946397 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.946397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.946452 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:12.946652 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:12.946653 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:12.946653 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:12.946762 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:12.946839 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:12.947002 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:46:12.947075 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:46:12.947659 29169 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:12.948020 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.947915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49160)
0927 18:46:12.947970 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.948000 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260927 18:46:12.948388 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.948293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49176)
0927 18:46:12.948340 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.948364 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:46:12.948486 29169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36577: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
I20260927 18:46:12.979635 29169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } attempt: 1
I20260927 18:46:12.980123 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.979987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49180)
0927 18:46:12.980056 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.980094 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:46:12.980510 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:12.980400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49196)
0927 18:46:12.980461 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:12.980485 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:46:12.980598 29169 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36577: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
W20260927 18:46:12.983166 29251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:12.983436 29251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:12.983486 29251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:12.984678 29251 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:12.984750 29251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:12.984776 29251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:12.984807 29251 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:12.984836 29251 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:12.986367 29251 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42327,127.15.110.253:35657,127.15.110.252:36577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.110.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:12.986663 29251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:12.986899 29251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:12.991858 29256 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:12.991973 29257 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:12.991859 29259 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:12.992096 29251 server_base.cc:1034] running on GCE node
I20260927 18:46:12.992389 29251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:12.992681 29251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:12.993832 29251 hybrid_clock.cc:648] HybridClock initialized: now 1790534772993823 us; error 32 us; skew 500 ppm
Sep 27 18:46:12 dist-test-slave-vs05 krb5kdc[29088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:12.996477 29251 init.cc:377] Logged in from keytab as kudu/127.15.110.252@KRBTEST.COM (short username kudu)
I20260927 18:46:12.996866 29251 webserver.cc:460] Webserver started at http://127.15.110.252:36913/ using document root <none> and password file <none>
I20260927 18:46:12.997089 29251 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:12.997136 29251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:12.997225 29251 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:12.998252 29251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data/instance:
uuid: "e075b9e88c4f47189563df814aa87c94"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "63749bfb5ce8b2075f60f576bd5c4ccc"
server_key_iv: "95fa4e3d6a3e3a860e8121df7915dfab"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.998555 29251 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal/instance:
uuid: "e075b9e88c4f47189563df814aa87c94"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "63749bfb5ce8b2075f60f576bd5c4ccc"
server_key_iv: "95fa4e3d6a3e3a860e8121df7915dfab"
server_key_version: "encryptionkey@0"
I20260927 18:46:12.999965 29251 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:46:13.000674 29266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:13.001004 29251 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:13.001130 29251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "e075b9e88c4f47189563df814aa87c94"
format_stamp: "Formatted at 2026-09-27 18:46:12 on dist-test-slave-vs05"
server_key: "63749bfb5ce8b2075f60f576bd5c4ccc"
server_key_iv: "95fa4e3d6a3e3a860e8121df7915dfab"
server_key_version: "encryptionkey@0"
I20260927 18:46:13.001221 29251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:13.002619 29244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } attempt: 2
I20260927 18:46:13.003150 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.003015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49208)
0927 18:46:13.003084 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.003120 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:46:13.003583 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.003497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49212)
0927 18:46:13.003542 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.003566 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:46:13.003696 29244 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36577: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36577: connect: Connection refused (error 111)
I20260927 18:46:13.019825 29251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:13.021814 29251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:13.021981 29251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:13.026093 29251 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.252:36577
I20260927 18:46:13.026129 29318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.252:36577 every 8 connection(s)
I20260927 18:46:13.026512 29251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
I20260927 18:46:13.027422 29319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:13.029021 29319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:46:13.030686 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29251
I20260927 18:46:13.030789 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790534620099873-15803-0/minicluster-data/master-2/wal/instance
I20260927 18:46:13.030992 15803 external_mini_cluster.cc:1428] Setting key 495eb1d176c2982d754adf5c977666e6
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:46:13.040191 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.029791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46830 (local address 127.15.110.254:42327)
0927 18:46:13.029862 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.029865 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.030032 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:46:13.030469 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.030470 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.030486 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.030536 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.031464 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.032089 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.033577 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.033779 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.033870 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.036399 (+ 2529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.036400 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.036400 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.036408 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.038659 (+ 2251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.039152 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.039154 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.039155 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.039236 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.039511 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.039512 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.039512 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.039633 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.039713 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.040016 (+ 303us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.040098 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:46:13.040408 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.029715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42327 (local address 127.0.0.1:46830)
0927 18:46:13.029895 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.029938 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:13.030389 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:13.030620 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:13.030625 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:13.031002 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:13.031362 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.031369 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.032193 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.032194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.033499 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.033502 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.033878 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.033879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.034016 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.034551 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:13.034566 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:13.036306 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:13.038743 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.038749 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.038750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.039066 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.039319 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.039320 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.039320 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.039420 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.039705 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:13.039711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:13.039870 (+ 159us) client_negotiation.cc:769] Sending connection context
0927 18:46:13.039984 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 18:46:13.040140 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"mutex_wait_us":40,"thread_start_us":77,"threads_started":1}
I20260927 18:46:13.040489 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.031387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49222 (local address 127.15.110.252:36577)
0927 18:46:13.031535 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.031540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.031712 (+ 172us) server_negotiation.cc:407] Connection header received
0927 18:46:13.031735 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.031737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.031769 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.031846 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.032399 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.033555 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.034295 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.034616 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.034701 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.037323 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.037331 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.037331 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.037338 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.039294 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.039627 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.039630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.039630 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.039688 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.039891 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.039892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.039892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.040040 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.040207 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.040254 (+ 47us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.040329 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260927 18:46:13.041136 29319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:13.049240 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.041361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35657 (local address 127.0.0.1:42752)
0927 18:46:13.041422 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.041432 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:13.041465 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:13.041668 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:13.041669 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:13.041780 (+ 111us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:13.041828 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.041829 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.042443 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.042443 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.042912 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.042913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.043429 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.043430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.043497 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.043902 (+ 405us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:13.043907 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:13.045395 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:13.048143 (+ 2748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.048144 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.048144 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.048366 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.048588 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.048589 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.048589 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.048653 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.048953 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:13.048954 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:13.049005 (+ 51us) client_negotiation.cc:769] Sending connection context
0927 18:46:13.049072 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:46:13.049150 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260927 18:46:13.049289 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.041442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42752 (local address 127.15.110.253:35657)
0927 18:46:13.041525 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.041528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.041537 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:46:13.041563 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.041564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.041574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.041638 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.041910 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.042378 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.043064 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.043344 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.043446 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.045511 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.045512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.045513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.045520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.048052 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.048469 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.048470 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.048470 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.048519 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.048725 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.048726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.048726 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.048880 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.048941 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.049115 (+ 174us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.049195 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:46:13.049597 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.041200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.15.110.254:42327)
0927 18:46:13.041263 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.041265 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.041273 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:13.041291 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.041292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.041306 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.041379 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.042452 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.042884 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.043633 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.043832 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.043914 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.046677 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.046679 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.046679 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.046686 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.048328 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.048725 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.048726 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.048727 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.048794 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.049023 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.049023 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.049024 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.049180 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.049372 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.049430 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.049510 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:46:13.050068 29319 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:46:13.055759 29169 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } attempt: 2
I20260927 18:46:13.058035 29321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.050305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49236)
0927 18:46:13.050348 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.050356 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:13.050386 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:13.050485 (+ 99us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:13.050486 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:13.050618 (+ 132us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:13.050644 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.050646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.051587 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.051587 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.052481 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.052484 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.052987 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.052987 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.053042 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.053480 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:13.053484 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:13.054812 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:13.056996 (+ 2184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.056997 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.056998 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.057209 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.057425 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.057426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.057426 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.057489 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.057786 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:13.057787 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:13.057819 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 18:46:13.057860 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 18:46:13.057939 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:46:13.058305 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.050351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49236 (local address 127.15.110.252:36577)
0927 18:46:13.050386 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.050388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.050396 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:13.050428 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.050429 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.050438 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.050485 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.050738 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.051509 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.052601 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.052908 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.052975 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.054904 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.054905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.054906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.054913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.056915 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.057292 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.057293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.057293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.057341 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.057568 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.057569 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.057569 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.057710 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.058130 (+ 420us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.058164 (+ 34us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.058235 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:46:13.058809 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.050231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42766 (local address 127.15.110.253:35657)
0927 18:46:13.050264 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.050266 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.050274 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:13.050332 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.050333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.050344 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.050397 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.050863 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.051594 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.052600 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.052895 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.052974 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.055453 (+ 2479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.055455 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.055455 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.055462 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.057220 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.057596 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.057597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.057598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.057649 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.057851 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.057852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.057853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.057967 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.058597 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.058640 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.058717 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:46:13.060114 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: Bootstrap starting.
I20260927 18:46:13.060325 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:46:13.060837 29319 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:13.061088 29319 log.cc:824] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:46:13.061952 29319 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: No bootstrap required, opened a new log
I20260927 18:46:13.064371 29319 raft_consensus.cc:348] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.064572 29319 raft_consensus.cc:374] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:13.064625 29319 raft_consensus.cc:729] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e075b9e88c4f47189563df814aa87c94, State: Initialized, Role: FOLLOWER
I20260927 18:46:13.064761 29319 consensus_queue.cc:260] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.065091 29319 sys_catalog.cc:564] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: configured and running, proceeding with master startup.
W20260927 18:46:13.066886 29344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:46:13.066985 29344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:46:13.068025 29171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.056327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49252)
0927 18:46:13.056382 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.056417 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:13.056459 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:13.056843 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:13.056844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:13.057013 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:13.057067 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.057069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.058448 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.058449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.059139 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.059142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.059649 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.059650 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.059708 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.060074 (+ 366us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:13.060078 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:13.061521 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:13.063470 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.063471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.063472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.063683 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.064001 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.064002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.064003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.064064 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.067682 (+ 3618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:13.067684 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:13.067741 (+ 57us) client_negotiation.cc:769] Sending connection context
0927 18:46:13.067798 (+ 57us) client_negotiation.cc:240] Negotiation successful
0927 18:46:13.067923 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:46:13.069357 29324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.056496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49252 (local address 127.15.110.252:36577)
0927 18:46:13.056694 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.056696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.056704 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:13.056723 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.056728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.056740 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.056814 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.057166 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.058352 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.059269 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.059557 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.059883 (+ 326us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.061623 (+ 1740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.061624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.061624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.061631 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.063368 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.063844 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.063845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.063845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.063924 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.064172 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.064173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.064173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.067580 (+ 3407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.069135 (+ 1555us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.069174 (+ 39us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.069256 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":115,"threads_started":1}
I20260927 18:46:13.067073 29251 master_runner.cc:186] Error getting master registration for 127.15.110.254:42327: OK, instance_id { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" instance_seqno: 1790534772711991 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:46:13.070111 29251 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:46:13.070850 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: Bootstrap starting.
I20260927 18:46:13.071350 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.071449 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.071619 29169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:13.071872 29169 log.cc:824] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:13.072772 29169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: No bootstrap required, opened a new log
I20260927 18:46:13.073992 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.063423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.15.110.252:36577)
0927 18:46:13.063461 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.063462 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.063773 (+ 311us) server_negotiation.cc:407] Connection header received
0927 18:46:13.063876 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.063877 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.063887 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.063916 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.064784 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.065529 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.066926 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.067229 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.067275 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.069998 (+ 2723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.070000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.070000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.070007 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.071584 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.072649 (+ 1065us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.072650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.072651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.072934 (+ 283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.073161 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.073161 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.073162 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.073532 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.073607 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.073831 (+ 224us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.073912 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:46:13.075583 29169 raft_consensus.cc:348] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.075681 29248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.063824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42776 (local address 127.15.110.253:35657)
0927 18:46:13.063862 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.063863 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.064274 (+ 411us) server_negotiation.cc:407] Connection header received
0927 18:46:13.064370 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.064371 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.064381 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.064412 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.065377 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.065977 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.068486 (+ 2509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.068779 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.068819 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.072460 (+ 3641us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.072461 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.072462 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.072469 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.074007 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.074791 (+ 784us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.074792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.074792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.074849 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.075051 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.075052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.075053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.075215 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.075310 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.075519 (+ 209us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.075601 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:46:13.075722 29169 raft_consensus.cc:374] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:13.075757 29169 raft_consensus.cc:729] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a7c2ef96cdb94522b3e835f0d05e6bf0, State: Initialized, Role: FOLLOWER
I20260927 18:46:13.076639 29169 consensus_queue.cc:260] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.076921 29347 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.076988 29169 sys_catalog.cc:564] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:13.077054 29347 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.077813 29172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.063755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46858 (local address 127.15.110.254:42327)
0927 18:46:13.063794 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.063796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.064120 (+ 324us) server_negotiation.cc:407] Connection header received
0927 18:46:13.064188 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.064189 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.064199 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.064244 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.064883 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.065575 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.066727 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.066997 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.067037 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.072178 (+ 5141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.072179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.072180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.072186 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.076544 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.076947 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.076948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.076949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.077009 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.077228 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.077229 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.077229 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.077361 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.077481 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.077643 (+ 162us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.077725 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 18:46:13.078958 29358 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:46:13.079042 29358 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:46:13.079710 29101 master_runner.cc:186] Error getting master registration for 127.15.110.253:35657: OK, instance_id { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" instance_seqno: 1790534772859306 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:46:13.081919 29101 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:46:13.093097 29347 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:13.093242 29347 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.093565 29347 leader_election.cc:288] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dd3ab5dcaef443fca9a773cadf6e700c (127.15.110.253:35657), e075b9e88c4f47189563df814aa87c94 (127.15.110.252:36577)
I20260927 18:46:13.093916 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e075b9e88c4f47189563df814aa87c94" is_pre_election: true
I20260927 18:46:13.094148 29293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7c2ef96cdb94522b3e835f0d05e6bf0 in term 0.
I20260927 18:46:13.094439 29117 leader_election.cc:302] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7c2ef96cdb94522b3e835f0d05e6bf0, e075b9e88c4f47189563df814aa87c94; no voters:
I20260927 18:46:13.094597 29347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:13.094689 29347 raft_consensus.cc:482] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:13.094722 29347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:13.094848 29244 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } attempt: 3
I20260927 18:46:13.095775 29347 raft_consensus.cc:504] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.095891 29347 leader_election.cc:288] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [CANDIDATE]: Term 1 election: Requested vote from peers dd3ab5dcaef443fca9a773cadf6e700c (127.15.110.253:35657), e075b9e88c4f47189563df814aa87c94 (127.15.110.252:36577)
I20260927 18:46:13.096223 29293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e075b9e88c4f47189563df814aa87c94"
I20260927 18:46:13.096310 29293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:13.097191 29293 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7c2ef96cdb94522b3e835f0d05e6bf0 in term 1.
I20260927 18:46:13.097702 29117 leader_election.cc:302] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a7c2ef96cdb94522b3e835f0d05e6bf0, e075b9e88c4f47189563df814aa87c94; no voters:
I20260927 18:46:13.097784 29347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:13.097940 29347 raft_consensus.cc:686] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [term 1 LEADER]: Becoming Leader. State: Replica: a7c2ef96cdb94522b3e835f0d05e6bf0, State: Running, Role: LEADER
I20260927 18:46:13.098037 29347 consensus_queue.cc:237] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.098546 29360 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c2ef96cdb94522b3e835f0d05e6bf0. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.098907 29360 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:13.099236 29362 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:13.099429 29362 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 27 18:46:13 dist-test-slave-vs05 krb5kdc[29088](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534772, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:46:13.100943 29293 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [term 1 FOLLOWER]: Refusing update from remote peer a7c2ef96cdb94522b3e835f0d05e6bf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:13.101251 29360 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:13.102499 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c2ef96cdb94522b3e835f0d05e6bf0. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.102581 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.102881 29246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.095084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36577 (local address 127.0.0.1:49272)
0927 18:46:13.095132 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:13.095146 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:13.095228 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:13.095386 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:13.095387 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:13.095573 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:13.095622 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.095624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.096507 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.096508 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.097313 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:13.097316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.097978 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.097979 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:13.098035 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.098502 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:13.098507 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:13.100018 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:13.101832 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.101833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.101834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.102043 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.102235 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:13.102236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:13.102236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:13.102288 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:13.102551 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:13.102552 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:13.102594 (+ 42us) client_negotiation.cc:769] Sending connection context
0927 18:46:13.102654 (+ 60us) client_negotiation.cc:240] Negotiation successful
0927 18:46:13.102747 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260927 18:46:13.103153 29323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:13.095146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49272 (local address 127.15.110.252:36577)
0927 18:46:13.095201 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:13.095203 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:13.095211 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:13.095302 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:13.095303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:13.095316 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:13.095383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:13.095692 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.096414 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.097551 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:13.097848 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:13.097908 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:13.100129 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:13.100131 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:13.100131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:13.100139 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:13.101743 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.102118 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.102119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.102119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.102171 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:13.102349 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:13.102350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:13.102350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:13.102468 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:13.102526 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:13.103000 (+ 474us) server_negotiation.cc:299] Negotiation successful
0927 18:46:13.103071 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:46:13.103837 29363 mvcc.cc:205] Tried to move back new op lower bound from 7334030430619004928 to 7334030430610030592. Current Snapshot: MvccSnapshot[applied={T|T < 7334030430619004928}]
I20260927 18:46:13.104591 29244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: Bootstrap starting.
I20260927 18:46:13.104655 29347 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.104743 29347 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:13.105216 29333 sys_catalog.cc:455] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.105298 29244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:13.105307 29333 sys_catalog.cc:458] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.105382 29364 mvcc.cc:205] Tried to move back new op lower bound from 7334030430619004928 to 7334030430610030592. Current Snapshot: MvccSnapshot[applied={T|T < 7334030430619004928}]
I20260927 18:46:13.105585 29244 log.cc:824] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:13.106480 29244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: No bootstrap required, opened a new log
I20260927 18:46:13.108566 29244 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.108714 29244 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:13.108758 29244 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd3ab5dcaef443fca9a773cadf6e700c, State: Initialized, Role: FOLLOWER
I20260927 18:46:13.108917 29244 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } }
I20260927 18:46:13.109230 29244 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:13.109287 29218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" is_pre_election: true
I20260927 18:46:13.109390 29218 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a7c2ef96cdb94522b3e835f0d05e6bf0 in term 0.
I20260927 18:46:13.109534 29216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:13.110949 29216 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 1 FOLLOWER]: Refusing update from remote peer a7c2ef96cdb94522b3e835f0d05e6bf0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:13.111137 29217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dd3ab5dcaef443fca9a773cadf6e700c"
I20260927 18:46:13.111693 29217 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a7c2ef96cdb94522b3e835f0d05e6bf0 in term 1.
I20260927 18:46:13.112298 29360 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:13.113986 29385 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a7c2ef96cdb94522b3e835f0d05e6bf0. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.114131 29385 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.114226 29385 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.114334 29385 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.109217 29370 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.114755 29370 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:46:13.115646 29362 catalog_manager.cc:1347] Generated new cluster ID: 8df0ca9d7a84490f87e6bb1561572cf0
I20260927 18:46:13.115698 29362 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:13.116735 29381 catalog_manager.cc:1259] Loaded cluster ID: 8df0ca9d7a84490f87e6bb1561572cf0
I20260927 18:46:13.116767 29381 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: loading cluster ID for follower catalog manager: success
W20260927 18:46:13.116849 29381 catalog_manager.cc:1576] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260927 18:46:13.116873 29381 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:46:13.121302 29367 sys_catalog.cc:455] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a7c2ef96cdb94522b3e835f0d05e6bf0" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42327 } } peers { permanent_uuid: "dd3ab5dcaef443fca9a773cadf6e700c" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35657 } } peers { permanent_uuid: "e075b9e88c4f47189563df814aa87c94" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36577 } } }
I20260927 18:46:13.121423 29367 sys_catalog.cc:458] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:13.122738 29362 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:46:13.123510 29362 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:13.128569 29362 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a7c2ef96cdb94522b3e835f0d05e6bf0: Generated new TSK 0
I20260927 18:46:13.128790 29362 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:46:14.067518 29344 catalog_manager.cc:1259] Loaded cluster ID: 8df0ca9d7a84490f87e6bb1561572cf0
I20260927 18:46:14.067595 29344 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: loading cluster ID for follower catalog manager: success
I20260927 18:46:14.068387 29344 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: acquiring CA information for follower catalog manager: success
I20260927 18:46:14.068652 29344 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e075b9e88c4f47189563df814aa87c94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:46:14.118019 29381 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: acquiring CA information for follower catalog manager: success
I20260927 18:46:14.118422 29381 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd3ab5dcaef443fca9a773cadf6e700c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:46:14.180670 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29101
I20260927 18:46:14.186252 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29176
I20260927 18:46:14.191309 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29251
2026-09-27T18:46:14Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2763 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:46:14 dist-test-slave-vs05 krb5kdc[29396](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:46:14 dist-test-slave-vs05 krb5kdc[29396](info): set up 2 sockets
Sep 27 18:46:14 dist-test-slave-vs05 krb5kdc[29396](info): commencing operation
krb5kdc: starting...
W20260927 18:46:15.243201 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:46:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:46:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:46:15.361429 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36949
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36949
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:15.447440 29409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:15.447682 29409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:15.447719 29409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:15.448844 29409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:15.448915 29409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:15.448942 29409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:15.448971 29409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:15.448988 29409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:15.450404 29409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:15.450677 29409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:15.450904 29409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:15.455879 29414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:15.455889 29417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:15.455889 29415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:15.456264 29409 server_base.cc:1034] running on GCE node
I20260927 18:46:15.456568 29409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:15.456866 29409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:15.458038 29409 hybrid_clock.cc:648] HybridClock initialized: now 1790534775458018 us; error 37 us; skew 500 ppm
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29396](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534775, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:15.460587 29409 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:15.461014 29409 webserver.cc:460] Webserver started at http://127.15.110.254:40505/ using document root <none> and password file <none>
I20260927 18:46:15.461238 29409 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:15.461306 29409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:15.461454 29409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:15.462509 29409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "528a3e085b48452fb34636b320f12599"
format_stamp: "Formatted at 2026-09-27 18:46:15 on dist-test-slave-vs05"
server_key: "6e1fb9bd6b5d75199917c4a9cb793325"
server_key_iv: "8ebc905495fdafa02b83d34c9d2e46c8"
server_key_version: "encryptionkey@0"
I20260927 18:46:15.462877 29409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "528a3e085b48452fb34636b320f12599"
format_stamp: "Formatted at 2026-09-27 18:46:15 on dist-test-slave-vs05"
server_key: "6e1fb9bd6b5d75199917c4a9cb793325"
server_key_iv: "8ebc905495fdafa02b83d34c9d2e46c8"
server_key_version: "encryptionkey@0"
I20260927 18:46:15.464291 29409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:46:15.465058 29424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:15.465255 29409 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:15.465354 29409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "528a3e085b48452fb34636b320f12599"
format_stamp: "Formatted at 2026-09-27 18:46:15 on dist-test-slave-vs05"
server_key: "6e1fb9bd6b5d75199917c4a9cb793325"
server_key_iv: "8ebc905495fdafa02b83d34c9d2e46c8"
server_key_version: "encryptionkey@0"
I20260927 18:46:15.465440 29409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:15.479485 29409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:15.481943 29409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:15.482098 29409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:15.486454 29409 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:36949
I20260927 18:46:15.486531 29476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:36949 every 8 connection(s)
I20260927 18:46:15.486864 29409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:15.487766 29477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:15.490018 29477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599: Bootstrap starting.
I20260927 18:46:15.490593 29477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:15.490869 29477 log.cc:824] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:15.491699 29477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599: No bootstrap required, opened a new log
I20260927 18:46:15.493243 29477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } }
I20260927 18:46:15.493405 29477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:15.493450 29477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 528a3e085b48452fb34636b320f12599, State: Initialized, Role: FOLLOWER
I20260927 18:46:15.493616 29477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } }
I20260927 18:46:15.493696 29477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:15.493738 29477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:15.493788 29477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:15.494302 29477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } }
I20260927 18:46:15.494413 29477 leader_election.cc:302] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 528a3e085b48452fb34636b320f12599; no voters:
I20260927 18:46:15.494590 29477 leader_election.cc:288] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:46:15.494632 29480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:15.494837 29477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:15.494885 29480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [term 1 LEADER]: Becoming Leader. State: Replica: 528a3e085b48452fb34636b320f12599, State: Running, Role: LEADER
I20260927 18:46:15.495002 29480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } }
I20260927 18:46:15.495558 29481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "528a3e085b48452fb34636b320f12599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } } }
I20260927 18:46:15.495651 29481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:15.495827 29482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 528a3e085b48452fb34636b320f12599. Latest consensus state: current_term: 1 leader_uuid: "528a3e085b48452fb34636b320f12599" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "528a3e085b48452fb34636b320f12599" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36949 } } }
I20260927 18:46:15.495894 29482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:15.496080 29489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:15.496357 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29409
I20260927 18:46:15.496486 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:46:15.496676 15803 external_mini_cluster.cc:1428] Setting key 4435939741775f33b33dee83e153190f
I20260927 18:46:15.496591 29489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:15.498210 29489 catalog_manager.cc:1347] Generated new cluster ID: d8462f0567794f31a756f056e17c48ce
I20260927 18:46:15.498261 29489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29396](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534775, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:15.506597 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:15.497100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38082 (local address 127.15.110.254:36949)
0927 18:46:15.497435 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:15.497444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:15.497476 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:46:15.497590 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:15.497597 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:15.497651 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:15.497785 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:15.498617 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:15.499674 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:15.500488 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:15.500906 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:15.501021 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:15.503398 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:15.503402 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:15.503402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:15.503413 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:15.505084 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:15.505461 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:15.505464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:15.505464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:15.505571 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:15.505775 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:15.505776 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:15.505776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:15.506017 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:15.506198 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:15.506277 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 18:46:15.506422 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":70,"threads_started":1}
I20260927 18:46:15.507511 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:46:15.515116 29489 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:46:15.515659 29489 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:15.519472 29489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 528a3e085b48452fb34636b320f12599: Generated new TSK 0
I20260927 18:46:15.519650 29489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:46:15.561394 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29409
2026-09-27T18:46:15Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1367 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29504](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29504](info): set up 2 sockets
Sep 27 18:46:15 dist-test-slave-vs05 krb5kdc[29504](info): commencing operation
krb5kdc: starting...
W20260927 18:46:16.624039 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:46:16 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:46:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:46:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:46:16.750644 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:16.832415 29517 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:16.832633 29517 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:16.832655 29517 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:16.833740 29517 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:16.833776 29517 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:16.833791 29517 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:16.833803 29517 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:16.833815 29517 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:16.835246 29517 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29517
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:16.835456 29517 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:16.835635 29517 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:16.840618 29522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:16.840601 29523 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:16.840606 29525 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:16.840761 29517 server_base.cc:1034] running on GCE node
I20260927 18:46:16.841143 29517 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:16.841483 29517 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:16.842660 29517 hybrid_clock.cc:648] HybridClock initialized: now 1790534776842651 us; error 51 us; skew 500 ppm
Sep 27 18:46:16 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534776, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:16.845230 29517 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:16.845712 29517 webserver.cc:460] Webserver started at http://127.15.110.254:43735/ using document root <none> and password file <none>
I20260927 18:46:16.845954 29517 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:16.846027 29517 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:16.846180 29517 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:16.847314 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "0c43e79639f4416b8a0a69be628e8a6d"
format_stamp: "Formatted at 2026-09-27 18:46:16 on dist-test-slave-vs05"
server_key: "2a6464f91d53540e76eeb8c4c8cb65ee"
server_key_iv: "63b8af01b326d15375f07c688231d9aa"
server_key_version: "encryptionkey@0"
I20260927 18:46:16.847683 29517 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "0c43e79639f4416b8a0a69be628e8a6d"
format_stamp: "Formatted at 2026-09-27 18:46:16 on dist-test-slave-vs05"
server_key: "2a6464f91d53540e76eeb8c4c8cb65ee"
server_key_iv: "63b8af01b326d15375f07c688231d9aa"
server_key_version: "encryptionkey@0"
I20260927 18:46:16.849103 29517 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:16.849906 29532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:16.850082 29517 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:16.850176 29517 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "0c43e79639f4416b8a0a69be628e8a6d"
format_stamp: "Formatted at 2026-09-27 18:46:16 on dist-test-slave-vs05"
server_key: "2a6464f91d53540e76eeb8c4c8cb65ee"
server_key_iv: "63b8af01b326d15375f07c688231d9aa"
server_key_version: "encryptionkey@0"
I20260927 18:46:16.850260 29517 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:16.867594 29517 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:16.869699 29517 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:16.869874 29517 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:16.874110 29517 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:38247
I20260927 18:46:16.874150 29584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:38247 every 8 connection(s)
I20260927 18:46:16.874514 29517 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:16.875748 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29517
I20260927 18:46:16.875805 29585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:16.875871 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:46:16.876098 15803 external_mini_cluster.cc:1428] Setting key 004e4ed337797e245cc492eee2e14fc4
I20260927 18:46:16.879505 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap starting.
I20260927 18:46:16.880250 29585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:16.880574 29585 log.cc:824] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:16.881494 29585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: No bootstrap required, opened a new log
Sep 27 18:46:16 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534776, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:16.883873 29585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:16.884061 29585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:16.884104 29585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Initialized, Role: FOLLOWER
I20260927 18:46:16.884268 29585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:16.884392 29585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:16.884447 29585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:16.884498 29585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:16.885195 29585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:16.885320 29585 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43e79639f4416b8a0a69be628e8a6d; no voters:
I20260927 18:46:16.885529 29585 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:46:16.885565 29590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:16.885679 29590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 LEADER]: Becoming Leader. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Running, Role: LEADER
I20260927 18:46:16.885820 29585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:16.885784 29590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:16.886188 29590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43e79639f4416b8a0a69be628e8a6d. Latest consensus state: current_term: 1 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:16.886291 29590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:16.886198 29591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:16.886469 29591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:16.886648 29596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:16.887213 29596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:16.888306 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:16.876694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44368 (local address 127.15.110.254:38247)
0927 18:46:16.876893 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:16.876903 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:16.876941 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:46:16.877522 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:16.877532 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:16.877609 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:16.877784 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:16.878358 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:16.879943 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:16.880884 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:16.881603 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:16.881729 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:16.884077 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:16.884081 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:16.884082 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:16.884095 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:16.886287 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:16.886751 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:16.886754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:16.886754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:16.886853 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:16.887079 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:16.887080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:16.887080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:16.887405 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:16.887590 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:16.888039 (+ 449us) server_negotiation.cc:299] Negotiation successful
0927 18:46:16.888162 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260927 18:46:16.889166 29607 mvcc.cc:205] Tried to move back new op lower bound from 7334030446133936128 to 7334030446124851200. Current Snapshot: MvccSnapshot[applied={T|T < 7334030446133936128}]
I20260927 18:46:16.889443 29596 catalog_manager.cc:1347] Generated new cluster ID: 1e736e9cbc1647d6bab92a33f1679fdc
I20260927 18:46:16.889488 29596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:16.908243 29596 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:46:16.909086 29596 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:16.921422 29596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Generated new TSK 0
I20260927 18:46:16.921654 29596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:46:16.952306 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:17.038322 29611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.038533 29611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.038558 29611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.039827 29611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.039875 29611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.039902 29611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:17.041397 29611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.041661 29611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.041917 29611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.047353 29616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.047446 29617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.047536 29611 server_base.cc:1034] running on GCE node
W20260927 18:46:17.047341 29619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.047874 29611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.048105 29611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.049254 29611 hybrid_clock.cc:648] HybridClock initialized: now 1790534777049206 us; error 64 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:17.051949 29611 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:17.052445 29611 webserver.cc:460] Webserver started at http://127.15.110.193:43463/ using document root <none> and password file <none>
I20260927 18:46:17.052729 29611 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:17.052814 29611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:17.052981 29611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:17.054198 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "fe19bb51900f46b9853524ed82e096fc"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "2a982271dd591e655d720ccfe2342ace"
server_key_iv: "53ab9e5d8b61d27be75cf2444be6c5e1"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.054550 29611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "fe19bb51900f46b9853524ed82e096fc"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "2a982271dd591e655d720ccfe2342ace"
server_key_iv: "53ab9e5d8b61d27be75cf2444be6c5e1"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.056015 29611 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:46:17.056735 29626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.056945 29611 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:46:17.057047 29611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "fe19bb51900f46b9853524ed82e096fc"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "2a982271dd591e655d720ccfe2342ace"
server_key_iv: "53ab9e5d8b61d27be75cf2444be6c5e1"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.057130 29611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:17.091267 29611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:17.093732 29611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:17.093916 29611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:17.094172 29611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:17.094491 29611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:17.094547 29611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.094587 29611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:17.094609 29611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.100862 29611 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46731
I20260927 18:46:17.100912 29739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46731 every 8 connection(s)
I20260927 18:46:17.101238 29611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:17.108294 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29611
I20260927 18:46:17.108429 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:46:17.108639 15803 external_mini_cluster.cc:1428] Setting key 00b2085bf773344f775826e5c81e00e4
I20260927 18:46:17.111387 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.102033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:46973 (local address 127.15.110.254:38247)
0927 18:46:17.102121 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.102124 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.102294 (+ 170us) server_negotiation.cc:407] Connection header received
0927 18:46:17.102854 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.102856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.102868 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.102932 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.103901 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.104427 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.105450 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.105664 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.105741 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.107993 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.107995 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.107995 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.108001 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.109738 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.110243 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.110244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.110244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.110298 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.110594 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.110595 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.110595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.110702 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.110797 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.111145 (+ 348us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.111263 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:46:17.111493 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.101923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:46973)
0927 18:46:17.102150 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.102199 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.102724 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.103007 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.103012 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.103602 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:17.103797 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.103802 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.104528 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.104529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.105336 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.105339 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.105747 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.105748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.105879 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.106416 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:17.106424 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:17.107891 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:17.109823 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.109831 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.109834 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.110164 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.110366 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.110367 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.110368 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.110525 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.110803 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:17.110811 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:17.110997 (+ 186us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.111099 (+ 102us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.111290 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":124,"threads_started":1}
I20260927 18:46:17.112031 29740 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:17.112141 29740 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:17.112316 29740 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:17.112908 29549 ts_manager.cc:194] Registered new tserver with Master: fe19bb51900f46b9853524ed82e096fc (127.15.110.193:46731)
I20260927 18:46:17.113894 29549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:46973
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:46:17.134938 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:17.217351 29745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.217598 29745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.217640 29745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.218822 29745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.218895 29745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.218941 29745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:17.220463 29745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.220736 29745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.220974 29745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.225997 29751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.226212 29745 server_base.cc:1034] running on GCE node
W20260927 18:46:17.225998 29753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.226054 29750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.226511 29745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.226737 29745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.227922 29745 hybrid_clock.cc:648] HybridClock initialized: now 1790534777227897 us; error 37 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:17.230522 29745 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:17.230995 29745 webserver.cc:460] Webserver started at http://127.15.110.194:44979/ using document root <none> and password file <none>
I20260927 18:46:17.231293 29745 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:17.231341 29745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:17.231495 29745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:17.232585 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "cb49ec3736054e30cbcebc876c0ac0b9"
server_key_iv: "390038888ce25a80a77b59fc42d647fd"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.232934 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "cb49ec3736054e30cbcebc876c0ac0b9"
server_key_iv: "390038888ce25a80a77b59fc42d647fd"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.234264 29745 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:46:17.235078 29760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.235280 29745 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:17.235378 29745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "cb49ec3736054e30cbcebc876c0ac0b9"
server_key_iv: "390038888ce25a80a77b59fc42d647fd"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.235473 29745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:17.244060 29745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:17.246086 29745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:17.246281 29745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:17.246500 29745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:17.246801 29745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:17.246860 29745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.246901 29745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:17.246934 29745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.252890 29745 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:36693
I20260927 18:46:17.252930 29873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:36693 every 8 connection(s)
I20260927 18:46:17.253242 29745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:17.259791 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29745
I20260927 18:46:17.259893 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:46:17.260092 15803 external_mini_cluster.cc:1428] Setting key e163c61d1c2f641ae1e496ad4620ea93
I20260927 18:46:17.263418 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.254030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:49445 (local address 127.15.110.254:38247)
0927 18:46:17.254096 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.254098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.254303 (+ 205us) server_negotiation.cc:407] Connection header received
0927 18:46:17.254770 (+ 467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.254771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.254781 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.254832 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.255863 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.256369 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.257426 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.257615 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.257669 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.260091 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.260092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.260093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.260099 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.261815 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.262192 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.262194 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.262194 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.262257 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.262657 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.262657 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.262658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.262779 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.262836 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.263270 (+ 434us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.263341 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:46:17.263649 29876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.254015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:49445)
0927 18:46:17.254183 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.254220 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.254689 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.254926 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.254930 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.255582 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:17.255775 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.255780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.256456 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.256457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.257344 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.257346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.257689 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.257690 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.257797 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.258404 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:17.258412 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:17.259978 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:17.261896 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.261902 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.261904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.262116 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.262380 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.262381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.262382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.262550 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.262869 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:17.262876 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:17.263048 (+ 172us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.263170 (+ 122us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.263382 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":83,"threads_started":1}
I20260927 18:46:17.264199 29874 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:17.264302 29874 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:17.264495 29874 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:17.264961 29549 ts_manager.cc:194] Registered new tserver with Master: 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:17.265516 29549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:49445
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:46:17.288044 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:17.374085 29879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.374348 29879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.374389 29879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.375634 29879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.375698 29879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.375746 29879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:17.377208 29879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.377483 29879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.377728 29879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.382789 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.382797 29887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.383111 29879 server_base.cc:1034] running on GCE node
W20260927 18:46:17.382797 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.383378 29879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.383628 29879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.384814 29879 hybrid_clock.cc:648] HybridClock initialized: now 1790534777384812 us; error 60 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:17.388103 29879 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:17.388604 29879 webserver.cc:460] Webserver started at http://127.15.110.195:39739/ using document root <none> and password file <none>
I20260927 18:46:17.388792 29879 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:17.388830 29879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:17.388928 29879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:17.389991 29879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "79bd32414f774c028892896ad03b6899"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "0dac6b1efae6fcd762a6699ea53c7820"
server_key_iv: "1fe392529df3432021bd9aac2e3eab08"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.390353 29879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "79bd32414f774c028892896ad03b6899"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "0dac6b1efae6fcd762a6699ea53c7820"
server_key_iv: "1fe392529df3432021bd9aac2e3eab08"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.391866 29879 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:46:17.392829 29894 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.393038 29879 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:17.393162 29879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "79bd32414f774c028892896ad03b6899"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "0dac6b1efae6fcd762a6699ea53c7820"
server_key_iv: "1fe392529df3432021bd9aac2e3eab08"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.393220 29879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:17.401485 29879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:17.403455 29879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:17.403566 29879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:17.403738 29879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:17.404043 29879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:17.404078 29879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.404099 29879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:17.404148 29879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.409790 29879 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:43749
I20260927 18:46:17.409873 30007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:43749 every 8 connection(s)
I20260927 18:46:17.410218 29879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:17.412994 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 29879
I20260927 18:46:17.413086 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:46:17.413249 15803 external_mini_cluster.cc:1428] Setting key 27864134d0ccd6fd488c43b48f16520a
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:17.420195 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.411102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:42025 (local address 127.15.110.254:38247)
0927 18:46:17.411168 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.411211 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.411407 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:46:17.411887 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.411888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.411898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.411955 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.412912 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.413605 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.414824 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.415029 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.415093 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.417459 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.417461 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.417461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.417469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.418951 (+ 1482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.419413 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.419414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.419414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.419473 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.419676 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.419676 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.419676 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.419775 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.419865 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.420066 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.420127 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:46:17.420450 30010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.411054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:42025)
0927 18:46:17.411271 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.411314 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.411804 (+ 490us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.412022 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.412025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.412605 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:17.412805 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.412809 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.413707 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.413708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.414740 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.414742 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.415115 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.415116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.415282 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.415848 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:17.415857 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:17.417333 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:17.419034 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.419040 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.419042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.419346 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.419533 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.419534 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.419534 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.419634 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.419841 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:17.419845 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:17.419963 (+ 118us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.420045 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.420167 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260927 18:46:17.420991 30008 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:17.421079 30008 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:17.421299 30008 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:17.421705 29549 ts_manager.cc:194] Registered new tserver with Master: 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749)
I20260927 18:46:17.422117 29549 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:42025
I20260927 18:46:17.425256 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:17.432597 29588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.426603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.15.110.254:38247)
0927 18:46:17.426679 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.426680 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.426724 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:46:17.426783 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.426784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.426792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.426837 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.427465 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.427957 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.428675 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.428852 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.428934 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.430102 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.430103 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.430103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.430110 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.431421 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.431725 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.431725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.431726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.431775 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.432011 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.432012 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.432013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.432172 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.432266 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.432373 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.432445 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:46:17.434442 29549 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44374:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:46:17.436118 29549 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:46:17.442888 30022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:36693 (local address 127.0.0.1:36506)
0927 18:46:17.439034 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.439042 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.439081 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.439400 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.439401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.439402 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:17.439457 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.439458 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440380 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.440381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.441712 (+ 1331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.441715 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442536 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.442616 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442631 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.442706 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.442790 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":15,"thread_start_us":78,"threads_started":1}
I20260927 18:46:17.442955 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39060 (local address 127.15.110.193:46731)
0927 18:46:17.438960 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.438965 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.438977 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:46:17.439021 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.439023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.439041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.439400 (+ 359us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:17.439473 (+ 73us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440603 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.441804 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442601 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442675 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442732 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.442826 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.442866 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":72,"threads_started":1}
I20260927 18:46:17.443001 30019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46731 (local address 127.0.0.1:39060)
0927 18:46:17.438780 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.438826 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.438903 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.439215 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.439219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.439222 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:17.439302 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.439304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440729 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.440730 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.441710 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.441713 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442686 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.442743 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442746 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.442815 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.442848 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":66,"threads_started":1}
I20260927 18:46:17.443202 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43749 (local address 127.0.0.1:51110)
0927 18:46:17.438920 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.438930 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.439018 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.439385 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.439385 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.439388 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:17.439445 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.439447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440711 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.440711 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.441825 (+ 1114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.441827 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442918 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.442990 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442994 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.443049 (+ 55us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.443093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":2,"thread_start_us":79,"threads_started":1}
I20260927 18:46:17.443395 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51110 (local address 127.15.110.195:43749)
0927 18:46:17.438873 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.438879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.438992 (+ 113us) server_negotiation.cc:407] Connection header received
0927 18:46:17.439157 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.439159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.439217 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.439276 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:17.439521 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440594 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.441895 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442601 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442677 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442732 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.443262 (+ 530us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.443313 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
I20260927 18:46:17.443890 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.438985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.15.110.194:36693)
0927 18:46:17.439240 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.439245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.439254 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:46:17.439293 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.439295 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.439309 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.439529 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:17.439601 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.440269 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.441828 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.442427 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.442538 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.442605 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.443760 (+ 1155us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.443811 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":70,"threads_started":1}
I20260927 18:46:17.444165 29942 tablet_service.cc:1467] Processing CreateTablet for tablet a43b351954b54b948afda8ad783a775e (DEFAULT_TABLE table=test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:17.444547 29942 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a43b351954b54b948afda8ad783a775e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:17.444514 29674 tablet_service.cc:1467] Processing CreateTablet for tablet a43b351954b54b948afda8ad783a775e (DEFAULT_TABLE table=test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:17.444679 29808 tablet_service.cc:1467] Processing CreateTablet for tablet a43b351954b54b948afda8ad783a775e (DEFAULT_TABLE table=test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:17.444818 29674 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a43b351954b54b948afda8ad783a775e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:17.445063 29808 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a43b351954b54b948afda8ad783a775e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:17.447930 30025 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Bootstrap starting.
I20260927 18:46:17.448136 30026 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Bootstrap starting.
I20260927 18:46:17.448269 30027 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Bootstrap starting.
I20260927 18:46:17.448632 30025 tablet_bootstrap.cc:654] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:17.448820 30026 tablet_bootstrap.cc:654] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:17.448851 30027 tablet_bootstrap.cc:654] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:17.449186 30025 log.cc:824] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:17.449429 30027 log.cc:824] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:17.449429 30026 log.cc:824] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:17.449968 30025 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: No bootstrap required, opened a new log
I20260927 18:46:17.450054 30025 ts_tablet_manager.cc:1397] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:17.450161 30026 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: No bootstrap required, opened a new log
I20260927 18:46:17.450168 30027 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: No bootstrap required, opened a new log
I20260927 18:46:17.450233 30027 ts_tablet_manager.cc:1397] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:46:17.450233 30026 ts_tablet_manager.cc:1397] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:17.451789 30025 raft_consensus.cc:348] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.451959 30025 raft_consensus.cc:374] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:17.452006 30025 raft_consensus.cc:729] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a1734a88bf4e479b3e7cc88f78a21c, State: Initialized, Role: FOLLOWER
I20260927 18:46:17.452147 30025 consensus_queue.cc:260] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.452189 30026 raft_consensus.cc:348] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.452189 30027 raft_consensus.cc:348] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.452369 30026 raft_consensus.cc:374] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:17.452415 30027 raft_consensus.cc:374] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:17.452411 30026 raft_consensus.cc:729] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Initialized, Role: FOLLOWER
I20260927 18:46:17.452445 30025 ts_tablet_manager.cc:1428] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:46:17.452458 30027 raft_consensus.cc:729] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd32414f774c028892896ad03b6899, State: Initialized, Role: FOLLOWER
I20260927 18:46:17.452590 29874 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
I20260927 18:46:17.452558 30026 consensus_queue.cc:260] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.452558 30027 consensus_queue.cc:260] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.452823 30027 ts_tablet_manager.cc:1428] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:46:17.452823 30026 ts_tablet_manager.cc:1428] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:46:17.453015 29740 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
I20260927 18:46:17.453073 30008 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
W20260927 18:46:17.503851 29875 tablet.cc:2369] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:17.572324 30033 raft_consensus.cc:482] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:17.572484 30033 raft_consensus.cc:504] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.572817 30033 leader_election.cc:288] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:17.575884 30024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.573227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:44747 (local address 127.15.110.194:36693)
0927 18:46:17.573291 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.573293 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.573303 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:46:17.573335 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.573336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.573349 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.573422 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:17.573578 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.574049 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575072 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.575528 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575619 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.575627 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.575781 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.575815 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:46:17.575881 30021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.573137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:39551 (local address 127.15.110.195:43749)
0927 18:46:17.573202 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.573204 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.573321 (+ 117us) server_negotiation.cc:407] Connection header received
0927 18:46:17.573399 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.573400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.573409 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.573481 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:17.573637 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.574102 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575091 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.575518 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575604 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.575612 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.575766 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.575812 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:46:17.575901 30034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.573065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43749 (local address 127.15.110.193:39551)
0927 18:46:17.573236 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.573247 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.573331 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.573483 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.573484 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.573486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:17.573565 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.573567 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.574187 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.574188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.574960 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.574962 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.575615 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.575690 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.575694 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.575762 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.575800 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":1,"thread_start_us":99,"threads_started":1}
I20260927 18:46:17.575902 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.573050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:36693 (local address 127.15.110.193:44747)
0927 18:46:17.573170 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.573184 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.573236 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.573425 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.573426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.573428 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:17.573504 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.573506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.574143 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.574144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.574972 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.574975 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.575618 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.575619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.575704 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.575707 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.575776 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.575815 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"mutex_wait_us":47}
I20260927 18:46:17.576239 29828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a43b351954b54b948afda8ad783a775e" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" is_pre_election: true
I20260927 18:46:17.576297 29962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a43b351954b54b948afda8ad783a775e" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32414f774c028892896ad03b6899" is_pre_election: true
I20260927 18:46:17.576442 29828 raft_consensus.cc:2454] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 0.
I20260927 18:46:17.576479 29962 raft_consensus.cc:2454] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 0.
I20260927 18:46:17.576684 29627 leader_election.cc:302] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48a1734a88bf4e479b3e7cc88f78a21c, fe19bb51900f46b9853524ed82e096fc; no voters:
I20260927 18:46:17.576857 30033 raft_consensus.cc:2781] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:17.576903 30033 raft_consensus.cc:482] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:17.576920 30033 raft_consensus.cc:3037] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:17.577593 30033 raft_consensus.cc:504] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.577741 30033 leader_election.cc:288] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 election: Requested vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:17.577924 29962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a43b351954b54b948afda8ad783a775e" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32414f774c028892896ad03b6899"
I20260927 18:46:17.577965 29828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a43b351954b54b948afda8ad783a775e" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
I20260927 18:46:17.578006 29962 raft_consensus.cc:3037] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:17.578053 29828 raft_consensus.cc:3037] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:17.578750 29828 raft_consensus.cc:2454] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 1.
I20260927 18:46:17.578745 29962 raft_consensus.cc:2454] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 1.
I20260927 18:46:17.578956 29630 leader_election.cc:302] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79bd32414f774c028892896ad03b6899, fe19bb51900f46b9853524ed82e096fc; no voters:
I20260927 18:46:17.579084 30033 raft_consensus.cc:2781] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:17.579280 30033 raft_consensus.cc:686] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 LEADER]: Becoming Leader. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Running, Role: LEADER
I20260927 18:46:17.579401 30033 consensus_queue.cc:237] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.580261 29549 catalog_manager.cc:5495] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc reported cstate change: term changed from 0 to 1, leader changed from <none> to fe19bb51900f46b9853524ed82e096fc (127.15.110.193). New cstate: current_term: 1 leader_uuid: "fe19bb51900f46b9853524ed82e096fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } health_report { overall_health: UNKNOWN } } }
I20260927 18:46:17.589064 30023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.586494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39064 (local address 127.15.110.193:46731)
0927 18:46:17.586535 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.586537 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.586545 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:17.586585 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.586586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.586601 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.586706 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:17.586917 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.587458 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.588294 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.588480 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.588532 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.588725 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:17.588818 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:17.588881 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.588977 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.588994 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:46:17.590754 29962 raft_consensus.cc:1261] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Refusing update from remote peer fe19bb51900f46b9853524ed82e096fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:17.591096 29828 raft_consensus.cc:1261] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Refusing update from remote peer fe19bb51900f46b9853524ed82e096fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:17.591092 30035 consensus_queue.cc:1035] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:17.591403 30035 consensus_queue.cc:1035] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:17.593611 30038 mvcc.cc:205] Tried to move back new op lower bound from 7334030449009053696 to 7334030448965804032. Current Snapshot: MvccSnapshot[applied={T|T < 7334030449009053696}]
I20260927 18:46:17.594169 30039 mvcc.cc:205] Tried to move back new op lower bound from 7334030449009053696 to 7334030448965804032. Current Snapshot: MvccSnapshot[applied={T|T < 7334030449009053696}]
I20260927 18:46:17.594838 30040 mvcc.cc:205] Tried to move back new op lower bound from 7334030449009053696 to 7334030448965804032. Current Snapshot: MvccSnapshot[applied={T|T < 7334030449009053696 or (T in {7334030449009053696})}]
I20260927 18:46:17.595992 29549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:17.596079 29549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44374:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:17.596761 29549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Sending DeleteTablet for 3 replicas of tablet a43b351954b54b948afda8ad783a775e
I20260927 18:46:17.597070 29674 tablet_service.cc:1514] Processing DeleteTablet for tablet a43b351954b54b948afda8ad783a775e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:17 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:39060
I20260927 18:46:17.597103 29942 tablet_service.cc:1514] Processing DeleteTablet for tablet a43b351954b54b948afda8ad783a775e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:17 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:51110
I20260927 18:46:17.597148 29808 tablet_service.cc:1514] Processing DeleteTablet for tablet a43b351954b54b948afda8ad783a775e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:17 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36506
I20260927 18:46:17.597311 30049 tablet_replica.cc:330] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: stopping tablet replica
I20260927 18:46:17.597409 30049 raft_consensus.cc:2229] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:17.597414 30048 tablet_replica.cc:330] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: stopping tablet replica
I20260927 18:46:17.597409 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29611
I20260927 18:46:17.597505 30049 raft_consensus.cc:2258] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:17.599252 30050 tablet_replica.cc:330] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: stopping tablet replica
I20260927 18:46:17.599346 30050 raft_consensus.cc:2229] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:17.599435 30050 raft_consensus.cc:2258] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:17.599529 30049 ts_tablet_manager.cc:1905] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:17.599865 30050 ts_tablet_manager.cc:1905] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:17.601233 30049 ts_tablet_manager.cc:1918] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:17.601313 30050 ts_tablet_manager.cc:1918] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:17.601338 30049 log.cc:1196] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/a43b351954b54b948afda8ad783a775e
I20260927 18:46:17.601380 30050 log.cc:1196] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/a43b351954b54b948afda8ad783a775e
I20260927 18:46:17.601605 30049 ts_tablet_manager.cc:1939] T a43b351954b54b948afda8ad783a775e P 79bd32414f774c028892896ad03b6899: Deleting consensus metadata
I20260927 18:46:17.601653 30050 ts_tablet_manager.cc:1939] T a43b351954b54b948afda8ad783a775e P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting consensus metadata
I20260927 18:46:17.602051 29536 catalog_manager.cc:4841] TS 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): tablet a43b351954b54b948afda8ad783a775e (table test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]) successfully deleted
I20260927 18:46:17.602156 29533 catalog_manager.cc:4841] TS 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): tablet a43b351954b54b948afda8ad783a775e (table test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]) successfully deleted
W20260927 18:46:17.603037 29533 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:46731 (error 108)
I20260927 18:46:17.603310 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29745
I20260927 18:46:17.604019 30020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.603447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46731 (local address 127.0.0.1:39072)
0927 18:46:17.603961 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.603994 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496}
W20260927 18:46:17.604144 29533 catalog_manager.cc:4584] TS fe19bb51900f46b9853524ed82e096fc (127.15.110.193:46731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a43b351954b54b948afda8ad783a775e: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46731: connect: Connection refused (error 111)
I20260927 18:46:17.608821 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29879
I20260927 18:46:17.613644 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 29517
I20260927 18:46:17.619463 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--webserver_interface=127.15.110.254
--webserver_port=43735
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:17.701967 30052 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.702339 30052 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.702365 30052 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.703727 30052 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:17.703771 30052 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.703786 30052 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.703799 30052 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:17.703857 30052 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:17.705402 30052 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30052
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.705669 30052 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.706015 30052 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.714768 30059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.714762 30061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.714854 30058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.715122 30052 server_base.cc:1034] running on GCE node
I20260927 18:46:17.715413 30052 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.715690 30052 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.716861 30052 hybrid_clock.cc:648] HybridClock initialized: now 1790534777716819 us; error 44 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:17.725129 30052 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:17.725637 30052 webserver.cc:460] Webserver started at http://127.15.110.254:43735/ using document root <none> and password file <none>
I20260927 18:46:17.725917 30052 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:17.725978 30052 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:17.727519 30052 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:17.728232 30068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.728456 30052 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:17.728603 30052 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "0c43e79639f4416b8a0a69be628e8a6d"
format_stamp: "Formatted at 2026-09-27 18:46:16 on dist-test-slave-vs05"
server_key: "2a6464f91d53540e76eeb8c4c8cb65ee"
server_key_iv: "63b8af01b326d15375f07c688231d9aa"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.729044 30052 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:17.749150 30052 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:17.751638 30052 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:17.751806 30052 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:17.756071 30052 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:38247
I20260927 18:46:17.756124 30120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:38247 every 8 connection(s)
I20260927 18:46:17.756552 30052 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:17.758153 30121 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:46:17.758858 30121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap starting.
I20260927 18:46:17.764999 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30052
I20260927 18:46:17.765328 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:46731
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=43463
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:17.767238 30121 log.cc:824] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:17.769145 30121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:17.769459 30121 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap complete.
I20260927 18:46:17.771863 30121 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:17.772126 30121 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Initialized, Role: FOLLOWER
I20260927 18:46:17.772338 30121 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:17.772440 30121 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:17.772501 30121 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:17.772550 30121 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:46:17.773617 30121 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:17.773736 30121 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43e79639f4416b8a0a69be628e8a6d; no voters:
I20260927 18:46:17.773938 30121 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:46:17.774012 30125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:46:17.774194 30125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 LEADER]: Becoming Leader. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Running, Role: LEADER
I20260927 18:46:17.774204 30121 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:17.774341 30125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:17.774780 30127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43e79639f4416b8a0a69be628e8a6d. Latest consensus state: current_term: 2 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:17.774945 30127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:17.775110 30126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:17.775216 30126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:17.775339 30133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:17.776157 30133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:17.776253 30133 catalog_manager.cc:1259] Loaded cluster ID: 1e736e9cbc1647d6bab92a33f1679fdc
I20260927 18:46:17.776288 30133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:17.776530 30133 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260927 18:46:17.781594 30133 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260927 18:46:17.784181 30133 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:17.784379 30133 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Loaded TSK: 0
I20260927 18:46:17.784588 30133 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:46:17.850777 30123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.850986 30123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.851011 30123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.852224 30123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.852270 30123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.852298 30123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:17.853766 30123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:46731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=43463
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.854015 30123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.854272 30123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.860339 30151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.860361 30149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.860358 30148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.860661 30123 server_base.cc:1034] running on GCE node
I20260927 18:46:17.860857 30123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.861116 30123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.862286 30123 hybrid_clock.cc:648] HybridClock initialized: now 1790534777862275 us; error 30 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:17.864758 30123 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:17.865243 30123 webserver.cc:460] Webserver started at http://127.15.110.193:43463/ using document root <none> and password file <none>
I20260927 18:46:17.865481 30123 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:17.865566 30123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:17.866958 30123 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:17.867771 30158 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:17.867991 30123 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:17.868093 30123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "fe19bb51900f46b9853524ed82e096fc"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "2a982271dd591e655d720ccfe2342ace"
server_key_iv: "53ab9e5d8b61d27be75cf2444be6c5e1"
server_key_version: "encryptionkey@0"
I20260927 18:46:17.868461 30123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:17.878260 30123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:17.880672 30123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:17.880844 30123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:17.881079 30123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:17.881577 30165 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:46:17.883016 30123 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:46:17.883066 30123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:46:17.883101 30123 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:46:17.883781 30123 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:46:17.883834 30123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:17.883868 30165 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Bootstrap starting.
I20260927 18:46:17.891731 30123 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46731
I20260927 18:46:17.891892 30272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46731 every 8 connection(s)
I20260927 18:46:17.892294 30123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:17.894950 30165 log.cc:824] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:17.897615 30165 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:17.898003 30165 tablet_bootstrap.cc:492] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Bootstrap complete.
I20260927 18:46:17.898990 30165 ts_tablet_manager.cc:1397] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.011s
I20260927 18:46:17.900228 30165 raft_consensus.cc:348] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.900556 30165 raft_consensus.cc:729] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Initialized, Role: FOLLOWER
I20260927 18:46:17.900650 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30123
I20260927 18:46:17.900701 30165 consensus_queue.cc:260] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } } peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } }
I20260927 18:46:17.900930 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:36693
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=44979
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:17.900991 30165 ts_tablet_manager.cc:1428] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:17.909183 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.893054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:54699)
0927 18:46:17.893325 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.893368 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.893962 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.896128 (+ 2166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.896133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.896480 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:17.896724 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.896730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.898390 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.898391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.899699 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.899702 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.902150 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.902152 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.902285 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.902882 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:17.902890 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:17.904727 (+ 1837us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:17.907308 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.907315 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.907317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.907619 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.907942 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.907943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.907943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.908065 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.908547 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:17.908552 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:17.908696 (+ 144us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.908799 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.908969 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":88,"threads_started":1}
I20260927 18:46:17.909652 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.893169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:54699 (local address 127.15.110.254:38247)
0927 18:46:17.895661 (+ 2492us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.895672 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.895717 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:46:17.895857 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.895864 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.895923 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.896083 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.896873 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.898257 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.899797 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.900406 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.902184 (+ 1778us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.904854 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.904859 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.904860 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.904874 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.907225 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.907716 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.907720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.907720 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.907859 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.908139 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.908140 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.908140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.908468 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.908568 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.908895 (+ 327us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.909054 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2437,"thread_start_us":94,"threads_started":1}
I20260927 18:46:17.910550 30273 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:17.910658 30273 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:17.910876 30273 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:17.911738 30085 ts_manager.cc:194] Registered new tserver with Master: fe19bb51900f46b9853524ed82e096fc (127.15.110.193:46731)
I20260927 18:46:17.912807 30085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:54699
I20260927 18:46:17.913784 30273 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.193@KRBTEST.COM
I20260927 18:46:17.921581 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.912601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39084 (local address 127.15.110.193:46731)
0927 18:46:17.912754 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:17.912758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:17.913304 (+ 546us) server_negotiation.cc:407] Connection header received
0927 18:46:17.913341 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:17.913343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:17.913358 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:17.913416 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:17.913953 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.914908 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.916123 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.916414 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.916471 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.918598 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:17.918606 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:17.918606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:17.918613 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:17.920215 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.920652 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.920656 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.920656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.920715 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:17.920953 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:17.920954 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:17.920955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:17.921085 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:17.921180 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:17.921384 (+ 204us) server_negotiation.cc:299] Negotiation successful
0927 18:46:17.921472 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":96,"threads_started":1}
I20260927 18:46:17.921602 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:17.912357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46731 (local address 127.0.0.1:39084)
0927 18:46:17.913121 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:17.913154 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:17.913223 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:17.913445 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:17.913449 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:17.913762 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:17.913820 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.913822 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.914986 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.914987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.916047 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:17.916050 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:17.916491 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:17.916492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:17.916567 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:17.917028 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:17.917033 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:17.918495 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:17.920321 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.920328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.920330 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.920550 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.920788 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:17.920789 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:17.920790 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:17.920855 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:17.921170 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:17.921176 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:17.921230 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 18:46:17.921366 (+ 136us) client_negotiation.cc:240] Negotiation successful
0927 18:46:17.921505 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":738,"thread_start_us":94,"threads_started":1}
I20260927 18:46:17.921991 30207 tablet_service.cc:1514] Processing DeleteTablet for tablet a43b351954b54b948afda8ad783a775e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:17 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:39084
I20260927 18:46:17.922699 30282 tablet_replica.cc:330] stopping tablet replica
I20260927 18:46:17.922943 30282 raft_consensus.cc:2229] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:17.923036 30282 raft_consensus.cc:2258] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:17.923389 30282 ts_tablet_manager.cc:1905] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:17.925556 30282 ts_tablet_manager.cc:1918] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:17.925631 30282 log.cc:1196] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/a43b351954b54b948afda8ad783a775e
I20260927 18:46:17.925905 30282 ts_tablet_manager.cc:1939] T a43b351954b54b948afda8ad783a775e P fe19bb51900f46b9853524ed82e096fc: Deleting consensus metadata
I20260927 18:46:17.926417 30069 catalog_manager.cc:4841] TS fe19bb51900f46b9853524ed82e096fc (127.15.110.193:46731): tablet a43b351954b54b948afda8ad783a775e (table test-table [id=7eea7e98a0ac4212b8e1931f99ec0db5]) successfully deleted
W20260927 18:46:17.988736 30279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:17.988945 30279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:17.988969 30279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:17.990123 30279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:17.990163 30279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:17.990190 30279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:17.991667 30279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:36693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=44979
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:17.991923 30279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:17.992147 30279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:17.997329 30290 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.997390 30287 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:17.997377 30288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:17.997529 30279 server_base.cc:1034] running on GCE node
I20260927 18:46:17.997792 30279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:17.998055 30279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:17.999233 30279 hybrid_clock.cc:648] HybridClock initialized: now 1790534777999210 us; error 29 us; skew 500 ppm
Sep 27 18:46:17 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:18.001782 30279 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:18.002168 30279 webserver.cc:460] Webserver started at http://127.15.110.194:44979/ using document root <none> and password file <none>
I20260927 18:46:18.002434 30279 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.002524 30279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.004051 30279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:18.004732 30297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.005049 30279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:18.005169 30279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "cb49ec3736054e30cbcebc876c0ac0b9"
server_key_iv: "390038888ce25a80a77b59fc42d647fd"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.005534 30279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.016270 30279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.018352 30279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.018533 30279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.018774 30279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:18.019081 30279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:18.019137 30279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.019198 30279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:18.019227 30279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.025355 30279 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:36693
I20260927 18:46:18.025398 30410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:36693 every 8 connection(s)
I20260927 18:46:18.025805 30279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:46:18.026394 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30279
I20260927 18:46:18.026679 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:43749
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=39739
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534777, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:18.037593 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.026638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:49485 (local address 127.15.110.254:38247)
0927 18:46:18.026694 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.026696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.026993 (+ 297us) server_negotiation.cc:407] Connection header received
0927 18:46:18.027869 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.027870 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.027881 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.027936 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:18.029090 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.029586 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.031297 (+ 1711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.031516 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.031627 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.034334 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:18.034335 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:18.034335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:18.034342 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:18.035944 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.036479 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.036480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.036480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.036540 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.036886 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.036887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.036887 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.037001 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:18.037053 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.037428 (+ 375us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.037505 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:46:18.037914 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.026606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:49485)
0927 18:46:18.026808 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.026898 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.027719 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.028046 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.028051 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.028710 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:18.028964 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.029004 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.029685 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.029686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.031136 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.031140 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.031617 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.031618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.031784 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.032351 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:18.032363 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:18.034223 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:18.036039 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:18.036047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:18.036050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:18.036394 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:18.036611 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:18.036612 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:18.036612 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:18.036768 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:18.037073 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:18.037080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:18.037271 (+ 191us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.037403 (+ 132us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.037609 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
I20260927 18:46:18.038501 30411 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:18.038597 30411 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:18.038801 30411 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:18.039283 30085 ts_manager.cc:194] Registered new tserver with Master: 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:18.039687 30085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:49485
W20260927 18:46:18.110530 30414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:18.110772 30414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:18.110813 30414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:18.112074 30414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:18.112145 30414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:18.112191 30414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:18.113627 30414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:43749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=39739
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:18.113883 30414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:18.114106 30414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:18.119092 30420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.119105 30419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.119306 30422 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.119463 30414 server_base.cc:1034] running on GCE node
I20260927 18:46:18.119666 30414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:18.119903 30414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:18.121073 30414 hybrid_clock.cc:648] HybridClock initialized: now 1790534778121069 us; error 36 us; skew 500 ppm
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:18.123594 30414 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:18.123968 30414 webserver.cc:460] Webserver started at http://127.15.110.195:39739/ using document root <none> and password file <none>
I20260927 18:46:18.124141 30414 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.124192 30414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.125473 30414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.126047 30429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.126235 30414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:18.126309 30414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "79bd32414f774c028892896ad03b6899"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "0dac6b1efae6fcd762a6699ea53c7820"
server_key_iv: "1fe392529df3432021bd9aac2e3eab08"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.126659 30414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.147699 30414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.149750 30414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.149919 30414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.150182 30414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:18.150516 30414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:18.150575 30414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.150616 30414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:18.150645 30414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.156364 30414 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:43749
I20260927 18:46:18.156445 30542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:43749 every 8 connection(s)
I20260927 18:46:18.156780 30414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:18.162387 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30414
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:18.166388 30276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.157542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:37369 (local address 127.15.110.254:38247)
0927 18:46:18.157626 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.157628 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.157804 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:46:18.158292 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.158293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.158303 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.158379 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:18.159291 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.159790 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.160745 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.160968 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.161024 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.163427 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:18.163428 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:18.163428 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:18.163434 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:18.164917 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.165364 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.165364 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.165365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.165421 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.165697 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.165697 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.165698 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.165817 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:18.165882 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.166198 (+ 316us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.166282 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260927 18:46:18.166567 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.157504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:37369)
0927 18:46:18.157685 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.157721 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.158203 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.158438 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.158441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.158987 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:18.159190 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.159195 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.159870 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.159871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.160670 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.160672 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.161045 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.161046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.161141 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.161584 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:18.161591 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:18.163328 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:18.164992 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:18.164998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:18.165000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:18.165293 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:18.165485 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:18.165486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:18.165486 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:18.165619 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:18.165914 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:18.165919 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:18.166060 (+ 141us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.166174 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.166339 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260927 18:46:18.167092 30543 heartbeater.cc:347] Connected to a master server at 127.15.110.254:38247
I20260927 18:46:18.167241 30543 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:18.167426 30543 heartbeater.cc:511] Master 127.15.110.254:38247 requested a full tablet report, sending...
I20260927 18:46:18.167806 30085 ts_manager.cc:194] Registered new tserver with Master: 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749)
I20260927 18:46:18.168254 30085 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:37369
I20260927 18:46:18.170079 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.162799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.15.110.254:38247)
0927 18:46:18.162906 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.162908 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.162956 (+ 48us) server_negotiation.cc:407] Connection header received
0927 18:46:18.163035 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.163036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.163046 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.163102 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:18.163846 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.164660 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.165700 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.165997 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.166043 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.167651 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:18.167652 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:18.167653 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:18.167660 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:18.169022 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.169345 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.169346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.169346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.169404 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:18.169570 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:18.169570 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:18.169571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:18.169669 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:18.169791 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.169866 (+ 75us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.169929 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":49,"threads_started":1}
I20260927 18:46:18.170855 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:18.173542 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.171226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44394 (local address 127.15.110.254:38247)
0927 18:46:18.171264 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.171265 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.171417 (+ 152us) server_negotiation.cc:407] Connection header received
0927 18:46:18.171481 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.171481 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.171507 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.171576 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:18.171692 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.172139 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.172925 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.173103 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.173152 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.173308 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:18.173390 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:18.173447 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.173483 (+ 36us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.173498 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:46:18.173827 30085 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:44394:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:46:18.175704 30085 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:46:18.177960 30207 tablet_service.cc:1467] Processing CreateTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 (DEFAULT_TABLE table=test-table [id=ff0b88d30ba04dd08b20c8a7e888c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:18.178190 30207 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237664de94ca452b8e8beccd2c9cf0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:18.179875 30165 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Bootstrap starting.
I20260927 18:46:18.180330 30165 tablet_bootstrap.cc:654] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:18.180984 30165 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: No bootstrap required, opened a new log
I20260927 18:46:18.181041 30165 ts_tablet_manager.cc:1397] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.181401 30165 raft_consensus.cc:348] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.181484 30165 raft_consensus.cc:374] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:18.181540 30165 raft_consensus.cc:729] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Initialized, Role: FOLLOWER
I20260927 18:46:18.181594 30165 consensus_queue.cc:260] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.181699 30165 ts_tablet_manager.cc:1428] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.182296 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.177536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:36693 (local address 127.0.0.1:36514)
0927 18:46:18.177873 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.177883 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.177949 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.178226 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.178227 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.178230 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:18.178296 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.178298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.179528 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.179529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.180740 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.180743 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.182007 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.182008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.182073 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.182088 (+ 15us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.182171 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.182220 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":86,"threads_started":1}
I20260927 18:46:18.182407 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.177658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.15.110.194:36693)
0927 18:46:18.177886 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.177890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.177936 (+ 46us) server_negotiation.cc:407] Connection header received
0927 18:46:18.178017 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.178019 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.178031 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.178134 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:18.178396 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.179406 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.180842 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.181827 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.181904 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.181974 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.182261 (+ 287us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.182311 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
I20260927 18:46:18.182703 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.177517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43749 (local address 127.0.0.1:51118)
0927 18:46:18.177564 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.177575 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.177612 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.179486 (+ 1874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.179487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.179489 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:18.179530 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.179531 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.180681 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.180682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.181560 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.181563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.182476 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.182477 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.182547 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.182550 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.182580 (+ 30us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.182618 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 18:46:18.183037 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.177670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51118 (local address 127.15.110.195:43749)
0927 18:46:18.179317 (+ 1647us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.179322 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.179333 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:18.179369 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.179371 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.179386 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.179471 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:18.179614 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.180564 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.181660 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.182378 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.182832 (+ 454us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.182889 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.182924 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.182963 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1612,"thread_start_us":106,"threads_started":1}
I20260927 18:46:18.183148 30345 tablet_service.cc:1467] Processing CreateTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 (DEFAULT_TABLE table=test-table [id=ff0b88d30ba04dd08b20c8a7e888c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:18.183542 30345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237664de94ca452b8e8beccd2c9cf0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:18.183772 30477 tablet_service.cc:1467] Processing CreateTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 (DEFAULT_TABLE table=test-table [id=ff0b88d30ba04dd08b20c8a7e888c8d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:18.184093 30477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 237664de94ca452b8e8beccd2c9cf0d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:18.187273 30553 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Bootstrap starting.
I20260927 18:46:18.187615 30554 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Bootstrap starting.
I20260927 18:46:18.188004 30553 tablet_bootstrap.cc:654] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:18.188458 30554 tablet_bootstrap.cc:654] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:18.188638 30553 log.cc:824] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:18.189073 30554 log.cc:824] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:18.189464 30553 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: No bootstrap required, opened a new log
I20260927 18:46:18.189519 30553 ts_tablet_manager.cc:1397] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:18.189888 30554 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: No bootstrap required, opened a new log
I20260927 18:46:18.189956 30554 ts_tablet_manager.cc:1397] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:18.191557 30553 raft_consensus.cc:348] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.191737 30553 raft_consensus.cc:374] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:18.191774 30553 raft_consensus.cc:729] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48a1734a88bf4e479b3e7cc88f78a21c, State: Initialized, Role: FOLLOWER
I20260927 18:46:18.191900 30553 consensus_queue.cc:260] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.191859 30554 raft_consensus.cc:348] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.192008 30554 raft_consensus.cc:374] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:18.192059 30554 raft_consensus.cc:729] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79bd32414f774c028892896ad03b6899, State: Initialized, Role: FOLLOWER
I20260927 18:46:18.192191 30553 ts_tablet_manager.cc:1428] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:46:18.192181 30554 consensus_queue.cc:260] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.192332 30411 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
I20260927 18:46:18.192497 30554 ts_tablet_manager.cc:1428] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:46:18.192503 30543 heartbeater.cc:503] Master 127.15.110.254:38247 was elected leader, sending a full tablet report...
I20260927 18:46:18.270146 30278 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:18.270336 30278 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.270758 30278 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:18.274909 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.271057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:48813 (local address 127.15.110.194:36693)
0927 18:46:18.271107 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.271108 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.271168 (+ 60us) server_negotiation.cc:407] Connection header received
0927 18:46:18.271260 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.271261 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.271270 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.271326 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:18.271545 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.272362 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.273912 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.274458 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.274546 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.274558 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.274796 (+ 238us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.274836 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:46:18.274907 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.270997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:46163 (local address 127.15.110.195:43749)
0927 18:46:18.271061 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.271063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.271153 (+ 90us) server_negotiation.cc:407] Connection header received
0927 18:46:18.271236 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.271237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.271248 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.271336 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:18.271545 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.272288 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.273914 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.274471 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.274573 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.274583 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.274784 (+ 201us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.274837 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:46:18.274914 30275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.270937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43749 (local address 127.15.110.193:46163)
0927 18:46:18.271042 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.271054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.271124 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.271340 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.271341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.271342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:18.271448 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.271452 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.272437 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.272438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.273777 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.273780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.274575 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.274576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.274714 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.274719 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.274777 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.274824 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":110}
I20260927 18:46:18.274998 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.270950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:36693 (local address 127.15.110.193:48813)
0927 18:46:18.271106 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.271114 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.271192 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.271340 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:18.271341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:18.271342 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:18.271467 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.271469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.272451 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.272451 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.273777 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:18.273780 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.274557 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.274558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:18.274632 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.274687 (+ 55us) client_negotiation.cc:769] Sending connection context
0927 18:46:18.274785 (+ 98us) client_negotiation.cc:240] Negotiation successful
0927 18:46:18.274836 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":35,"thread_start_us":89,"threads_started":1}
I20260927 18:46:18.275384 30365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237664de94ca452b8e8beccd2c9cf0d0" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" is_pre_election: true
I20260927 18:46:18.275555 30365 raft_consensus.cc:2454] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 0.
I20260927 18:46:18.275578 30497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237664de94ca452b8e8beccd2c9cf0d0" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32414f774c028892896ad03b6899" is_pre_election: true
I20260927 18:46:18.275763 30159 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48a1734a88bf4e479b3e7cc88f78a21c, fe19bb51900f46b9853524ed82e096fc; no voters:
I20260927 18:46:18.275789 30497 raft_consensus.cc:2454] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 0.
I20260927 18:46:18.275918 30278 raft_consensus.cc:2781] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:18.275970 30278 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:18.275986 30278 raft_consensus.cc:3037] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 0 FOLLOWER]: Advancing to term 1
W20260927 18:46:18.276331 30412 tablet.cc:2369] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:18.276888 30278 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.277055 30278 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 election: Requested vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
I20260927 18:46:18.277187 30497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237664de94ca452b8e8beccd2c9cf0d0" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79bd32414f774c028892896ad03b6899"
I20260927 18:46:18.277204 30365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "237664de94ca452b8e8beccd2c9cf0d0" candidate_uuid: "fe19bb51900f46b9853524ed82e096fc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
I20260927 18:46:18.277292 30497 raft_consensus.cc:3037] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:18.277288 30365 raft_consensus.cc:3037] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:18.277978 30365 raft_consensus.cc:2454] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 1.
I20260927 18:46:18.278142 30497 raft_consensus.cc:2454] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe19bb51900f46b9853524ed82e096fc in term 1.
I20260927 18:46:18.278146 30159 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 48a1734a88bf4e479b3e7cc88f78a21c, fe19bb51900f46b9853524ed82e096fc; no voters:
I20260927 18:46:18.278231 30278 raft_consensus.cc:2781] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:18.278388 30278 raft_consensus.cc:686] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 LEADER]: Becoming Leader. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Running, Role: LEADER
I20260927 18:46:18.278481 30278 consensus_queue.cc:237] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.279328 30084 catalog_manager.cc:5495] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc reported cstate change: term changed from 0 to 1, leader changed from <none> to fe19bb51900f46b9853524ed82e096fc (127.15.110.193). New cstate: current_term: 1 leader_uuid: "fe19bb51900f46b9853524ed82e096fc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } health_report { overall_health: HEALTHY } } }
I20260927 18:46:18.302271 30281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.299817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39088 (local address 127.15.110.193:46731)
0927 18:46:18.299894 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:18.299896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:18.299908 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:46:18.299944 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:18.299945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:18.299963 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:18.300047 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:18.300162 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.300667 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.301423 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:18.301643 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:18.301725 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:18.301894 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:18.302024 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:18.302116 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:18.302156 (+ 40us) server_negotiation.cc:299] Negotiation successful
0927 18:46:18.302180 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:46:18.303960 30497 raft_consensus.cc:1261] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Refusing update from remote peer fe19bb51900f46b9853524ed82e096fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:18.303959 30365 raft_consensus.cc:1261] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Refusing update from remote peer fe19bb51900f46b9853524ed82e096fc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:18.304360 30278 consensus_queue.cc:1035] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:18.304534 30560 consensus_queue.cc:1035] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [LEADER]: Connected to new peer: Peer: permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:18.306807 30562 mvcc.cc:205] Tried to move back new op lower bound from 7334030451930251264 to 7334030451829305344. Current Snapshot: MvccSnapshot[applied={T|T < 7334030451930251264}]
I20260927 18:46:18.307027 30565 mvcc.cc:205] Tried to move back new op lower bound from 7334030451930251264 to 7334030451829305344. Current Snapshot: MvccSnapshot[applied={T|T < 7334030451930251264}]
I20260927 18:46:18.307245 30563 mvcc.cc:205] Tried to move back new op lower bound from 7334030451930251264 to 7334030451829305344. Current Snapshot: MvccSnapshot[applied={T|T < 7334030451930251264 or (T in {7334030451930251264})}]
I20260927 18:46:18.309589 30084 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:44394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:18.309686 30084 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:44394:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:18.310420 30084 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Sending DeleteTablet for 3 replicas of tablet 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:18.310761 30207 tablet_service.cc:1514] Processing DeleteTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:18 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:39084
I20260927 18:46:18.310756 30345 tablet_service.cc:1514] Processing DeleteTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:18 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:36514
I20260927 18:46:18.310787 30477 tablet_service.cc:1514] Processing DeleteTablet for tablet 237664de94ca452b8e8beccd2c9cf0d0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:18 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:51118
I20260927 18:46:18.310856 30282 tablet_replica.cc:330] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: stopping tablet replica
I20260927 18:46:18.310914 30282 raft_consensus.cc:2229] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:46:18.311012 30282 raft_consensus.cc:2258] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:18.311043 30573 tablet_replica.cc:330] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: stopping tablet replica
I20260927 18:46:18.311136 30573 raft_consensus.cc:2229] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:18.311146 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30123
I20260927 18:46:18.311254 30573 raft_consensus.cc:2258] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:18.311659 30573 ts_tablet_manager.cc:1905] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:18.312028 30572 tablet_replica.cc:330] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: stopping tablet replica
I20260927 18:46:18.312135 30572 raft_consensus.cc:2229] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:18.312251 30572 raft_consensus.cc:2258] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:18.313513 30573 ts_tablet_manager.cc:1918] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:18.313591 30573 log.cc:1196] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:18.313872 30573 ts_tablet_manager.cc:1939] T 237664de94ca452b8e8beccd2c9cf0d0 P 79bd32414f774c028892896ad03b6899: Deleting consensus metadata
I20260927 18:46:18.314544 30572 ts_tablet_manager.cc:1905] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:18.315586 30072 catalog_manager.cc:4841] TS 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): tablet 237664de94ca452b8e8beccd2c9cf0d0 (table test-table [id=ff0b88d30ba04dd08b20c8a7e888c8d8]) successfully deleted
I20260927 18:46:18.316458 30572 ts_tablet_manager.cc:1918] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:18.316534 30572 log.cc:1196] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:18.316817 30572 ts_tablet_manager.cc:1939] T 237664de94ca452b8e8beccd2c9cf0d0 P 48a1734a88bf4e479b3e7cc88f78a21c: Deleting consensus metadata
I20260927 18:46:18.317276 30069 catalog_manager.cc:4841] TS 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): tablet 237664de94ca452b8e8beccd2c9cf0d0 (table test-table [id=ff0b88d30ba04dd08b20c8a7e888c8d8]) successfully deleted
W20260927 18:46:18.318554 30069 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:46731 (error 108)
I20260927 18:46:18.318801 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30279
I20260927 18:46:18.319487 30280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.318963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:46731 (local address 127.0.0.1:39098)
0927 18:46:18.319422 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.319460 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441}
W20260927 18:46:18.319617 30069 catalog_manager.cc:4584] TS fe19bb51900f46b9853524ed82e096fc (127.15.110.193:46731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 237664de94ca452b8e8beccd2c9cf0d0: Network error: Client connection negotiation failed: client connection to 127.15.110.193:46731: connect: Connection refused (error 111)
I20260927 18:46:18.324714 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30414
I20260927 18:46:18.329548 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30052
I20260927 18:46:18.335043 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--webserver_interface=127.15.110.254
--webserver_port=43735
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:38247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:18.418793 30575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:18.419009 30575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:18.419034 30575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:18.420403 30575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:18.420460 30575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:18.420476 30575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:18.420490 30575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:18.420501 30575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:18.422087 30575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:38247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:38247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=43735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:18.422307 30575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:18.422482 30575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:18.427675 30580 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.427675 30583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.427917 30575 server_base.cc:1034] running on GCE node
W20260927 18:46:18.427696 30581 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.428257 30575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:18.428565 30575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:18.429742 30575 hybrid_clock.cc:648] HybridClock initialized: now 1790534778429726 us; error 46 us; skew 500 ppm
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:18.432364 30575 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:18.432935 30575 webserver.cc:460] Webserver started at http://127.15.110.254:43735/ using document root <none> and password file <none>
I20260927 18:46:18.433192 30575 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.433240 30575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.434664 30575 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.435729 30590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.435945 30575 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.436036 30575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "0c43e79639f4416b8a0a69be628e8a6d"
format_stamp: "Formatted at 2026-09-27 18:46:16 on dist-test-slave-vs05"
server_key: "2a6464f91d53540e76eeb8c4c8cb65ee"
server_key_iv: "63b8af01b326d15375f07c688231d9aa"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.436383 30575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.449138 30575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.451632 30575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.451743 30575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.455581 30575 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:38247
I20260927 18:46:18.455611 30642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:38247 every 8 connection(s)
I20260927 18:46:18.455989 30575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:18.457585 30643 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:46:18.458293 30643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap starting.
I20260927 18:46:18.459941 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30575
I20260927 18:46:18.460304 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:46731
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=43463
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:18.467775 30643 log.cc:824] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:18.470306 30643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:18.470646 30643 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d: Bootstrap complete.
I20260927 18:46:18.473162 30643 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:18.473438 30643 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Initialized, Role: FOLLOWER
I20260927 18:46:18.473639 30643 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:18.473728 30643 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:18.473784 30643 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:18.473855 30643 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:46:18.474921 30643 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:18.475052 30643 leader_election.cc:302] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c43e79639f4416b8a0a69be628e8a6d; no voters:
I20260927 18:46:18.475353 30643 leader_election.cc:288] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:46:18.475389 30647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:46:18.475507 30647 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [term 3 LEADER]: Becoming Leader. State: Replica: 0c43e79639f4416b8a0a69be628e8a6d, State: Running, Role: LEADER
I20260927 18:46:18.475608 30647 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } }
I20260927 18:46:18.475651 30643 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:18.475870 30648 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:18.475872 30647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c43e79639f4416b8a0a69be628e8a6d. Latest consensus state: current_term: 3 leader_uuid: "0c43e79639f4416b8a0a69be628e8a6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c43e79639f4416b8a0a69be628e8a6d" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 38247 } } }
I20260927 18:46:18.476058 30648 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:18.476078 30647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c43e79639f4416b8a0a69be628e8a6d [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:18.476711 30654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:18.477725 30654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:18.477851 30654 catalog_manager.cc:1259] Loaded cluster ID: 1e736e9cbc1647d6bab92a33f1679fdc
I20260927 18:46:18.477900 30654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260927 18:46:18.478343 30654 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f47955289ad google::LogMessage::Fail() at ??:0
@ 0x7f479552b88a google::LogMessage::SendToLog() at ??:0
@ 0x7f47955284de google::LogMessage::Flush() at ??:0
@ 0x7f4795529a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f479d4490f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f479d4528b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f47973f60d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f47973eb954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f479824f6db start_thread at ??:0
@ 0x7f479402661f clone at ??:0
W20260927 18:46:18.557716 30645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:18.557931 30645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:18.557956 30645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:18.559147 30645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:18.559247 30645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:18.559296 30645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:18.560782 30645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:46731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=43463
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:18.561055 30645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:18.561300 30645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:18.567534 30679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.567621 30680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.567526 30682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.568225 30645 server_base.cc:1034] running on GCE node
I20260927 18:46:18.568444 30645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:18.568722 30645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:18.569882 30645 hybrid_clock.cc:648] HybridClock initialized: now 1790534778569842 us; error 56 us; skew 500 ppm
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:18.572592 30645 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:18.573056 30645 webserver.cc:460] Webserver started at http://127.15.110.193:43463/ using document root <none> and password file <none>
I20260927 18:46:18.573297 30645 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.573367 30645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.574802 30645 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.575662 30689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.575872 30645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:18.576012 30645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "fe19bb51900f46b9853524ed82e096fc"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "2a982271dd591e655d720ccfe2342ace"
server_key_iv: "53ab9e5d8b61d27be75cf2444be6c5e1"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.576380 30645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.588409 30645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.590874 30645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.591053 30645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.591352 30645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:18.591804 30696 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:46:18.593801 30645 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:46:18.593887 30645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:46:18.593926 30645 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:46:18.594573 30645 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:46:18.594638 30645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:18.594741 30696 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Bootstrap starting.
I20260927 18:46:18.601888 30645 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:46731
I20260927 18:46:18.601958 30803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:46731 every 8 connection(s)
I20260927 18:46:18.602344 30645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:18.605336 30696 log.cc:824] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:18.606189 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30645
I20260927 18:46:18.606492 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:36693
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=44979
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:18.607836 30696 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:18.608170 30696 tablet_bootstrap.cc:492] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Bootstrap complete.
I20260927 18:46:18.609411 30696 ts_tablet_manager.cc:1397] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.008s
I20260927 18:46:18.610427 30696 raft_consensus.cc:348] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.610684 30696 raft_consensus.cc:729] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe19bb51900f46b9853524ed82e096fc, State: Initialized, Role: FOLLOWER
I20260927 18:46:18.610865 30696 consensus_queue.cc:260] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:18.611142 30696 ts_tablet_manager.cc:1428] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
W20260927 18:46:18.692956 30808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:18.693183 30808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:18.693208 30808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:18.694398 30808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:18.694442 30808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:18.694468 30808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:18.696007 30808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:36693
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=44979
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:18.696257 30808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:18.696497 30808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:18.701704 30814 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.701951 30817 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.701924 30808 server_base.cc:1034] running on GCE node
W20260927 18:46:18.702277 30815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.702536 30808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:18.702829 30808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:18.704002 30808 hybrid_clock.cc:648] HybridClock initialized: now 1790534778703982 us; error 33 us; skew 500 ppm
W20260927 18:46:18.706664 30806 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:46:18.603123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:45743)
0927 18:46:18.605540 (+ 2417us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.605590 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:18.606232 (+ 642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:18.706452 (+100220us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":2366,"thread_start_us":120,"threads_started":1}
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260927 18:46:18.707042 30804 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:38247 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:38247: Client connection negotiation failed: client connection to 127.15.110.254:38247: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:46:18.707235 30808 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:18.707449 30806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.707264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:42925)
0927 18:46:18.707350 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.707397 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260927 18:46:18.707768 30806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.707665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:48885)
0927 18:46:18.707721 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.707746 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:46:18.707772 30808 webserver.cc:460] Webserver started at http://127.15.110.194:44979/ using document root <none> and password file <none>
W20260927 18:46:18.707870 30804 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:46:18.708020 30808 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.708097 30808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.709458 30808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.710129 30824 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.710326 30808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:18.710430 30808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "48a1734a88bf4e479b3e7cc88f78a21c"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "cb49ec3736054e30cbcebc876c0ac0b9"
server_key_iv: "390038888ce25a80a77b59fc42d647fd"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.710850 30808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.721436 30808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.723492 30808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.723663 30808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.723906 30808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:18.724220 30808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:18.724278 30808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.724340 30808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:18.724368 30808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.730448 30808 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:36693
I20260927 18:46:18.730492 30937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:36693 every 8 connection(s)
I20260927 18:46:18.730836 30808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:46:18.731860 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.731475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:54399)
0927 18:46:18.731638 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.731721 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":75,"threads_started":1}
W20260927 18:46:18.732285 30938 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:38247 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:38247: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
I20260927 18:46:18.732594 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.732497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:60691)
0927 18:46:18.732548 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.732576 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:46:18.732863 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.732813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:39833)
0927 18:46:18.732835 (+ 22us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.732852 (+ 17us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13}
W20260927 18:46:18.732976 30938 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:46:18.734109 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30808
I20260927 18:46:18.734335 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:43749
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:38247
--webserver_port=39739
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:42851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:18.817010 30941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:18.817232 30941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:18.817257 30941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:18.818373 30941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:18.818415 30941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:18.818442 30941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:18.819923 30941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:43749
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=39739
--tserver_master_addrs=127.15.110.254:38247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:18.820119 30941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:18.820293 30941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:18.825397 30949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.825387 30946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:18.825433 30947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:18.825686 30941 server_base.cc:1034] running on GCE node
I20260927 18:46:18.825932 30941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:18.826164 30941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:18.827291 30941 hybrid_clock.cc:648] HybridClock initialized: now 1790534778827268 us; error 44 us; skew 500 ppm
Sep 27 18:46:18 dist-test-slave-vs05 krb5kdc[29504](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:18.829957 30941 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:18.830440 30941 webserver.cc:460] Webserver started at http://127.15.110.195:39739/ using document root <none> and password file <none>
I20260927 18:46:18.830672 30941 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:18.830744 30941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:18.832173 30941 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:18.832782 30956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.833058 30941 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:18.833160 30941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "79bd32414f774c028892896ad03b6899"
format_stamp: "Formatted at 2026-09-27 18:46:17 on dist-test-slave-vs05"
server_key: "0dac6b1efae6fcd762a6699ea53c7820"
server_key_iv: "1fe392529df3432021bd9aac2e3eab08"
server_key_version: "encryptionkey@0"
I20260927 18:46:18.833524 30941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:18.844455 30941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:18.846846 30941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:18.847050 30941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:18.847360 30941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:18.847689 30941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:18.847751 30941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.847796 30941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:18.847823 30941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:18.853976 30941 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:43749
I20260927 18:46:18.854017 31069 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:43749 every 8 connection(s)
I20260927 18:46:18.854372 30941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:18.855504 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.855040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:50849)
0927 18:46:18.855249 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.855337 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260927 18:46:18.855909 31070 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:38247 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:38247: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
I20260927 18:46:18.856218 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.856115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:45447)
0927 18:46:18.856169 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.856199 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:46:18.856518 31072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:18.856426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:51709)
0927 18:46:18.856472 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:18.856495 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:46:18.856618 31070 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:46:18.859447 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 30941
I20260927 18:46:19.708622 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:19.708378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:46547)
0927 18:46:19.708522 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:19.708562 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":89,"threads_started":1}
I20260927 18:46:19.733575 31075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:19.733294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:34751)
0927 18:46:19.733489 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:19.733528 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":53,"threads_started":1}
I20260927 18:46:19.857105 31076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:19.856914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:32981)
0927 18:46:19.857022 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:19.857055 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":52,"threads_started":1}
I20260927 18:46:20.083112 31077 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:20.083359 31077 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:20.083752 31077 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
Sep 27 18:46:20 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.195@KRBTEST.COM
Sep 27 18:46:20 dist-test-slave-vs05 krb5kdc[29504](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534778, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.194@KRBTEST.COM
I20260927 18:46:20.094372 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.084036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:45599 (local address 127.15.110.195:43749)
0927 18:46:20.084191 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:20.084203 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:20.084238 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:46:20.084686 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:20.084688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:20.084730 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:20.084966 (+ 236us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:20.085639 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.086662 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.087786 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.088202 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.088336 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:20.090641 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:20.090646 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:20.090646 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:20.090658 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:20.092722 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093147 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:20.093150 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:20.093151 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:20.093250 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093505 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:20.093506 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:20.093507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:20.093751 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:20.093847 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:20.094129 (+ 282us) server_negotiation.cc:299] Negotiation successful
0927 18:46:20.094252 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":50,"threads_started":1}
I20260927 18:46:20.094362 31078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.083929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43749 (local address 127.15.110.193:45599)
0927 18:46:20.084095 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:20.084105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:20.084163 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:20.084939 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:20.084940 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:20.085321 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:20.085504 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:20.085509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.086768 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.086770 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:20.087695 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:20.087697 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.088303 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.088305 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:20.088422 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:20.089066 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:20.089071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:20.090522 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:20.092817 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:20.092824 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:20.092827 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:20.093045 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:20.093323 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:20.093325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:20.093435 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:20.093826 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:20.093833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:20.093968 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 18:46:20.094079 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:46:20.094254 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":52,"thread_start_us":77,"threads_started":1}
I20260927 18:46:20.094460 31074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.083927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:36693 (local address 127.15.110.193:56759)
0927 18:46:20.083983 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:20.083995 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:20.084052 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:20.084920 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:20.084923 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:20.085321 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:20.085504 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:20.085510 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.086786 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.086787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:20.087694 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:20.087697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.088301 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.088302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:20.088422 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:20.089010 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:20.089018 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:20.090976 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:20.093206 (+ 2230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:20.093207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:20.093379 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:20.093632 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:20.093633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:20.093699 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:20.094069 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:20.094071 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:20.094115 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:46:20.094181 (+ 66us) client_negotiation.cc:240] Negotiation successful
0927 18:46:20.094321 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":71}
I20260927 18:46:20.094828 31080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.084079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:56759 (local address 127.15.110.194:36693)
0927 18:46:20.084221 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:20.084231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:20.084268 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:46:20.084695 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:20.084697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:20.084733 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:20.084881 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:20.085632 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.086661 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.087827 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:20.088202 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:20.088328 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:20.091097 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:20.091100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:20.091101 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:20.091112 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:20.093111 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093465 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:20.093468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:20.093468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:20.093560 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:20.093765 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:20.093766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:20.093767 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:20.093993 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:20.094086 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:20.094255 (+ 169us) server_negotiation.cc:299] Negotiation successful
0927 18:46:20.094382 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97,"thread_start_us":64,"threads_started":1}
W20260927 18:46:20.095294 30693 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
W20260927 18:46:20.095554 30690 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:20.095619 30690 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe19bb51900f46b9853524ed82e096fc; no voters: 48a1734a88bf4e479b3e7cc88f78a21c, 79bd32414f774c028892896ad03b6899
I20260927 18:46:20.095726 31077 raft_consensus.cc:2726] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:46:20.709476 31081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.709207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:40337)
0927 18:46:20.709380 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:20.709418 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":102,"threads_started":1}
I20260927 18:46:20.734282 31082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.734057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:57609)
0927 18:46:20.734201 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:20.734241 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":105,"threads_started":1}
I20260927 18:46:20.857903 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:20.857599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:36685)
0927 18:46:20.857795 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:20.857843 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260927 18:46:21.710235 31084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:21.709984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:44841)
0927 18:46:21.710144 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:21.710187 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":67,"threads_started":1}
I20260927 18:46:21.734915 31085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:21.734715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:57225)
0927 18:46:21.734835 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:21.734874 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":59,"threads_started":1}
I20260927 18:46:21.858709 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:21.858439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:53623)
0927 18:46:21.858615 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:21.858662 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":94,"threads_started":1}
I20260927 18:46:22.344961 31087 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:22.345100 31087 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:22.345327 31087 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
W20260927 18:46:22.345729 30693 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
W20260927 18:46:22.345880 30690 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:22.345961 30690 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe19bb51900f46b9853524ed82e096fc; no voters: 48a1734a88bf4e479b3e7cc88f78a21c, 79bd32414f774c028892896ad03b6899
I20260927 18:46:22.346061 31087 raft_consensus.cc:2726] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:46:22.711161 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:22.710855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:57717)
0927 18:46:22.711040 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:22.711091 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":94,"threads_started":1}
I20260927 18:46:22.735613 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:22.735402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:55229)
0927 18:46:22.735532 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:22.735569 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260927 18:46:22.859604 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:22.859347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:43787)
0927 18:46:22.859514 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:22.859559 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":73,"threads_started":1}
I20260927 18:46:23.712144 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:23.711905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:57859)
0927 18:46:23.712052 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:23.712094 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":65,"threads_started":1}
I20260927 18:46:23.736430 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:23.736191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:46731)
0927 18:46:23.736331 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:23.736383 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":59,"threads_started":1}
I20260927 18:46:23.860378 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:23.860149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:49713)
0927 18:46:23.860291 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:23.860330 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":71,"threads_started":1}
I20260927 18:46:24.713029 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:24.712721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:52237)
0927 18:46:24.712892 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:24.712970 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":65,"threads_started":1}
I20260927 18:46:24.737267 31095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:24.737016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:40941)
0927 18:46:24.737135 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:24.737171 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260927 18:46:24.861084 31096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:24.860830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:59147)
0927 18:46:24.860995 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:24.861038 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
I20260927 18:46:24.967378 31097 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:24.967519 31097 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:24.967736 31097 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
W20260927 18:46:24.968106 30690 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
W20260927 18:46:24.968184 30693 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:24.968334 30693 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe19bb51900f46b9853524ed82e096fc; no voters: 48a1734a88bf4e479b3e7cc88f78a21c, 79bd32414f774c028892896ad03b6899
I20260927 18:46:24.968410 31097 raft_consensus.cc:2726] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:46:25.713967 31098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:25.713656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:44277)
0927 18:46:25.713871 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:25.713919 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
I20260927 18:46:25.738204 31099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:25.737968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:46323)
0927 18:46:25.738098 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:25.738144 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
I20260927 18:46:25.861901 31100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:25.861634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:36037)
0927 18:46:25.861793 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:25.861846 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":78,"threads_started":1}
I20260927 18:46:26.533828 31101 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:26.533967 31101 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:26.534212 31101 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
W20260927 18:46:26.534570 30690 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
W20260927 18:46:26.534662 30693 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:26.534725 30693 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe19bb51900f46b9853524ed82e096fc; no voters: 48a1734a88bf4e479b3e7cc88f78a21c, 79bd32414f774c028892896ad03b6899
I20260927 18:46:26.534799 31101 raft_consensus.cc:2726] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:46:26.714843 31102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:26.714587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:37129)
0927 18:46:26.714759 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:26.714796 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260927 18:46:26.738960 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:26.738735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:35293)
0927 18:46:26.738860 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:26.738894 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":68,"threads_started":1}
I20260927 18:46:26.862874 31104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:26.862591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:42745)
0927 18:46:26.862760 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:26.862814 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":98,"threads_started":1}
I20260927 18:46:27.715843 31105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:27.715609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:43173)
0927 18:46:27.715737 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:27.715783 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":61,"threads_started":1}
I20260927 18:46:27.739672 31106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:27.739439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:46695)
0927 18:46:27.739589 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:27.739626 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":106,"threads_started":1}
I20260927 18:46:27.863715 31107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:27.863475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:42283)
0927 18:46:27.863621 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:27.863670 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260927 18:46:28.716584 31108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:28.716340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:60095)
0927 18:46:28.716494 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:28.716537 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":71,"threads_started":1}
I20260927 18:46:28.740417 31109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:28.740195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:55409)
0927 18:46:28.740339 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:28.740373 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
I20260927 18:46:28.864468 31110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:28.864195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:49155)
0927 18:46:28.864375 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:28.864419 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":83,"threads_started":1}
I20260927 18:46:29.717535 31111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:29.717245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:56531)
0927 18:46:29.717412 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:29.717469 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":75,"threads_started":1}
I20260927 18:46:29.741091 31112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:29.740892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:38593)
0927 18:46:29.741006 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:29.741044 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":61,"threads_started":1}
I20260927 18:46:29.865309 31113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:29.865050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:39513)
0927 18:46:29.865185 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:29.865229 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":62,"threads_started":1}
I20260927 18:46:30.718313 31114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:30.718094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:59003)
0927 18:46:30.718238 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:30.718272 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":89,"threads_started":1}
I20260927 18:46:30.741806 31115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:30.741597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:48351)
0927 18:46:30.741716 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:30.741758 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":60,"threads_started":1}
I20260927 18:46:30.866117 31116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:30.865857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:43871)
0927 18:46:30.866025 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:30.866069 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":68,"threads_started":1}
I20260927 18:46:31.719120 31117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:31.718860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:49657)
0927 18:46:31.719028 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:31.719073 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":71,"threads_started":1}
I20260927 18:46:31.742530 31118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:31.742295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:43645)
0927 18:46:31.742453 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:31.742485 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":102,"threads_started":1}
I20260927 18:46:31.866948 31119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:31.866701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:50233)
0927 18:46:31.866841 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:31.866901 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":63,"threads_started":1}
I20260927 18:46:31.917307 31120 raft_consensus.cc:482] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:31.917431 31120 raft_consensus.cc:504] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "79bd32414f774c028892896ad03b6899" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43749 } } peers { permanent_uuid: "48a1734a88bf4e479b3e7cc88f78a21c" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 36693 } } peers { permanent_uuid: "fe19bb51900f46b9853524ed82e096fc" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 46731 } }
I20260927 18:46:31.917662 31120 leader_election.cc:288] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749), 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693)
W20260927 18:46:31.918081 30693 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 79bd32414f774c028892896ad03b6899 (127.15.110.195:43749): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
W20260927 18:46:31.918185 30690 leader_election.cc:341] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 48a1734a88bf4e479b3e7cc88f78a21c (127.15.110.194:36693): Not found: Tablet not found: 237664de94ca452b8e8beccd2c9cf0d0
I20260927 18:46:31.918227 30690 leader_election.cc:302] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fe19bb51900f46b9853524ed82e096fc; no voters: 48a1734a88bf4e479b3e7cc88f78a21c, 79bd32414f774c028892896ad03b6899
I20260927 18:46:31.918298 31120 raft_consensus.cc:2726] T 237664de94ca452b8e8beccd2c9cf0d0 P fe19bb51900f46b9853524ed82e096fc [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:46:32.719935 31121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:32.719693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:56691)
0927 18:46:32.719845 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:32.719889 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":72,"threads_started":1}
I20260927 18:46:32.743170 31122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:32.742977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:33631)
0927 18:46:32.743092 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:32.743124 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":63,"threads_started":1}
I20260927 18:46:32.867686 31123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:32.867473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:37127)
0927 18:46:32.867598 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:32.867640 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260927 18:46:33.720746 31124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:33.720530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.193:42465)
0927 18:46:33.720665 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:33.720700 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260927 18:46:33.743952 31125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:33.743723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.194:45131)
0927 18:46:33.743864 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:33.743907 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":76,"threads_started":1}
I20260927 18:46:33.862646 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30645
I20260927 18:46:33.867666 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30808
I20260927 18:46:33.869226 31126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:33.868947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:38247 (local address 127.15.110.195:57675)
0927 18:46:33.869130 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:33.869166 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:38247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":68,"threads_started":1}
I20260927 18:46:33.872489 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 30941
2026-09-27T18:46:33Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18312 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:46:33 dist-test-slave-vs05 krb5kdc[31129](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:46:33 dist-test-slave-vs05 krb5kdc[31129](info): set up 2 sockets
Sep 27 18:46:33 dist-test-slave-vs05 krb5kdc[31129](info): commencing operation
krb5kdc: starting...
W20260927 18:46:34.907495 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.015s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:46:34 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:46:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:46:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:46:35.034396 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:35.118453 31142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:35.118659 31142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:35.118680 31142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:35.119906 31142 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:46:35.119951 31142 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:46:35.119966 31142 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:46:35.119976 31142 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:35.119989 31142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:35.120002 31142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:35.120015 31142 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:35.120064 31142 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:35.121539 31142 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:35.121790 31142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:35.122022 31142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:35.126921 31147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.127068 31142 server_base.cc:1034] running on GCE node
W20260927 18:46:35.126906 31148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.126906 31150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.127411 31142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:35.127705 31142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:35.128891 31142 hybrid_clock.cc:648] HybridClock initialized: now 1790534795128871 us; error 41 us; skew 500 ppm
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:35.131528 31142 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:35.131944 31142 webserver.cc:460] Webserver started at http://127.15.110.254:44427/ using document root <none> and password file <none>
I20260927 18:46:35.132192 31142 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:35.132265 31142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:35.132421 31142 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:35.133522 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "a74443fcc1794ad7ac753bc220b9f853"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "1ed24ffa557b4999331f2daa46e35262"
server_key_iv: "b94521313ac72d796f5a60847134bffa"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.133898 31142 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "a74443fcc1794ad7ac753bc220b9f853"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "1ed24ffa557b4999331f2daa46e35262"
server_key_iv: "b94521313ac72d796f5a60847134bffa"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.135408 31142 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:46:35.136201 31157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.136380 31142 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:46:35.136489 31142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "a74443fcc1794ad7ac753bc220b9f853"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "1ed24ffa557b4999331f2daa46e35262"
server_key_iv: "b94521313ac72d796f5a60847134bffa"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.136566 31142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:35.146618 31142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:35.148816 31142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:35.148981 31142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:35.153102 31142 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:42323
I20260927 18:46:35.153136 31209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:42323 every 8 connection(s)
I20260927 18:46:35.153527 31142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:35.154475 31210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:35.156879 31210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap starting.
I20260927 18:46:35.157464 31210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:35.157768 31210 log.cc:824] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:35.158568 31210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: No bootstrap required, opened a new log
I20260927 18:46:35.159467 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31142
I20260927 18:46:35.159583 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:46:35.159785 15803 external_mini_cluster.cc:1428] Setting key 34f865d07f5163b3193507806cc97848
I20260927 18:46:35.160651 31210 raft_consensus.cc:348] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:35.160813 31210 raft_consensus.cc:374] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:35.160905 31210 raft_consensus.cc:729] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Initialized, Role: FOLLOWER
I20260927 18:46:35.161087 31210 consensus_queue.cc:260] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:35.161187 31210 raft_consensus.cc:388] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:35.161231 31210 raft_consensus.cc:482] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:35.161280 31210 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:35.162110 31210 raft_consensus.cc:504] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:35.162228 31210 leader_election.cc:302] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a74443fcc1794ad7ac753bc220b9f853; no voters:
I20260927 18:46:35.162424 31210 leader_election.cc:288] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:46:35.162452 31215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:35.162557 31215 raft_consensus.cc:686] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 LEADER]: Becoming Leader. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Running, Role: LEADER
I20260927 18:46:35.162691 31210 sys_catalog.cc:564] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:35.162712 31215 consensus_queue.cc:237] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:35.163204 31216 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a74443fcc1794ad7ac753bc220b9f853. Latest consensus state: current_term: 1 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:46:35.163328 31216 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:35.163527 31217 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:46:35.163656 31217 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
W20260927 18:46:35.164726 31231 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:46:35.164810 31231 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:46:35.164866 31222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:35.165010 31222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:35.166715 31222 catalog_manager.cc:1347] Generated new cluster ID: 73eec58f55e74331b6e92a9182058715
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:35.166769 31222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:35.170677 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.160351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56588 (local address 127.15.110.254:42323)
0927 18:46:35.160555 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.160563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.160589 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:46:35.160685 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.160690 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.160733 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.160874 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:35.161616 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.162790 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.163890 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.164374 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.164597 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.167278 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:35.167282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:35.167282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:35.167295 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:35.169067 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.169458 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.169462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.169462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.169571 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.169794 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.169795 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.169795 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.170052 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:35.170209 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.170324 (+ 115us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.170476 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":119,"threads_started":1}
I20260927 18:46:35.184698 31222 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:46:35.185300 31222 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:35.188935 31222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Generated new TSK 0
I20260927 18:46:35.189153 31222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:46:35.224092 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:35.306593 31236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:35.306813 31236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:35.306836 31236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:35.308068 31236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:35.308111 31236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:35.308138 31236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:35.309603 31236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:35.309898 31236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:35.310144 31236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:35.315302 31241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.315327 31244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.315472 31236 server_base.cc:1034] running on GCE node
W20260927 18:46:35.315342 31242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.315800 31236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:35.316064 31236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:35.317220 31236 hybrid_clock.cc:648] HybridClock initialized: now 1790534795317205 us; error 28 us; skew 500 ppm
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:35.320057 31236 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:35.320520 31236 webserver.cc:460] Webserver started at http://127.15.110.193:33057/ using document root <none> and password file <none>
I20260927 18:46:35.320752 31236 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:35.320832 31236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:35.320983 31236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:35.322045 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "2236d8bd113a49e79c10857b4ee157c7"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "8ef1fa9985a52b922c40b755a8490f80"
server_key_iv: "63921fd3650fa9e17d9478290d717169"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.322412 31236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "2236d8bd113a49e79c10857b4ee157c7"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "8ef1fa9985a52b922c40b755a8490f80"
server_key_iv: "63921fd3650fa9e17d9478290d717169"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.323966 31236 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260927 18:46:35.324837 31251 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.325057 31236 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:35.325165 31236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "2236d8bd113a49e79c10857b4ee157c7"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "8ef1fa9985a52b922c40b755a8490f80"
server_key_iv: "63921fd3650fa9e17d9478290d717169"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.325241 31236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:35.344010 31236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:35.346251 31236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:35.346424 31236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:35.346671 31236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:35.346985 31236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:35.347041 31236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.347101 31236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:35.347131 31236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.352996 31236 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:44351
I20260927 18:46:35.353099 31364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:44351 every 8 connection(s)
I20260927 18:46:35.353410 31236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:35.359148 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31236
I20260927 18:46:35.359304 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:46:35.359498 15803 external_mini_cluster.cc:1428] Setting key a4dbd0b3af8f01b8066a9d7f826325aa
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:35.363626 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.354210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:40645 (local address 127.15.110.254:42323)
0927 18:46:35.354283 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.354285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.354461 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:46:35.354934 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.354935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.354944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.355009 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:35.356089 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.356599 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.357596 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.357815 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.357881 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.360406 (+ 2525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:35.360407 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:35.360408 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:35.360418 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:35.361982 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.362431 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.362432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.362433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.362492 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.362779 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.362780 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.362781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.362934 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:35.363002 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.363384 (+ 382us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.363481 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:46:35.363770 31367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.354160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:40645)
0927 18:46:35.354328 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.354371 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.354839 (+ 468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.355065 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.355069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.355758 (+ 689us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:35.355967 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.355972 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.356688 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.356689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.357497 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.357499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.357921 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.357922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.358039 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.358531 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:35.358540 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:35.360301 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:35.362096 (+ 1795us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.362103 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.362106 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.362348 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.362570 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.362571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.362572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.362687 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.363029 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:35.363035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:35.363165 (+ 130us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.363350 (+ 185us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.363545 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
I20260927 18:46:35.364559 31365 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:35.364677 31365 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:35.364873 31365 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:35.365415 31174 ts_manager.cc:194] Registered new tserver with Master: 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:35.366060 31174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:40645
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:46:35.387456 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:35.469897 31370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:35.470104 31370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:35.470127 31370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:35.471318 31370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:35.471359 31370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:35.471385 31370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:35.472869 31370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:35.473061 31370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:35.473243 31370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:35.478478 31375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.478487 31376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.478487 31378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.478554 31370 server_base.cc:1034] running on GCE node
I20260927 18:46:35.479079 31370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:35.479353 31370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:35.480522 31370 hybrid_clock.cc:648] HybridClock initialized: now 1790534795480509 us; error 29 us; skew 500 ppm
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:35.483047 31370 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:35.483565 31370 webserver.cc:460] Webserver started at http://127.15.110.194:38397/ using document root <none> and password file <none>
I20260927 18:46:35.483790 31370 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:35.483862 31370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:35.484023 31370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:35.485080 31370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "d890352b11a24b1aab0c1ea6d71c928e"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "7b597afda578c175b25f170670fc8ea0"
server_key_iv: "f630c0a5539e8f60bf5844f15da3d3e4"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.485448 31370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "d890352b11a24b1aab0c1ea6d71c928e"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "7b597afda578c175b25f170670fc8ea0"
server_key_iv: "f630c0a5539e8f60bf5844f15da3d3e4"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.486817 31370 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:46:35.487686 31385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.487936 31370 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:35.488036 31370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "d890352b11a24b1aab0c1ea6d71c928e"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "7b597afda578c175b25f170670fc8ea0"
server_key_iv: "f630c0a5539e8f60bf5844f15da3d3e4"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.488117 31370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:35.495776 31370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:35.497923 31370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:35.498096 31370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:35.498324 31370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:35.498631 31370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:35.498667 31370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.498688 31370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:35.498736 31370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.504688 31370 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:33281
I20260927 18:46:35.504767 31498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:33281 every 8 connection(s)
I20260927 18:46:35.505093 31370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:35.512516 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31370
I20260927 18:46:35.512627 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:46:35.512883 15803 external_mini_cluster.cc:1428] Setting key 517350d78f52eb5f98753d2c5ad6a48a
I20260927 18:46:35.514834 31501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.505977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:50931)
0927 18:46:35.506169 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.506206 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.506676 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.506966 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.506969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.507616 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:35.507814 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.507819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.508489 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.508490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.509319 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.509321 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.509664 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.509664 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.509762 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.510348 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:35.510357 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:35.511778 (+ 1421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:35.513474 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.513483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.513489 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.513775 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.513977 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.513977 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.513978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.514075 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.514355 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:35.514360 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:35.514471 (+ 111us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.514552 (+ 81us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.514671 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260927 18:46:35.514876 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.505991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:50931 (local address 127.15.110.254:42323)
0927 18:46:35.506060 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.506062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.506291 (+ 229us) server_negotiation.cc:407] Connection header received
0927 18:46:35.506814 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.506815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.506825 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.506882 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:35.507903 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.508411 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.509393 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.509592 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.509645 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.511891 (+ 2246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:35.511892 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:35.511892 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:35.511899 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:35.513399 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.513848 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.513849 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.513849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.513910 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.514136 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.514137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.514138 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.514284 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:35.514353 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.514593 (+ 240us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.514686 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:46:35.515473 31499 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:35.515597 31499 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:35.515796 31499 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:35.516232 31174 ts_manager.cc:194] Registered new tserver with Master: d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
I20260927 18:46:35.516661 31174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:50931
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:46:35.538856 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:35.620904 31504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:35.621150 31504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:35.621188 31504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:35.622357 31504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:35.622429 31504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:35.622471 31504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:35.623975 31504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:35.624259 31504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:35.624498 31504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:35.629769 31509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.629750 31512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:35.629750 31510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:35.630066 31504 server_base.cc:1034] running on GCE node
I20260927 18:46:35.630261 31504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:35.630491 31504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:35.631673 31504 hybrid_clock.cc:648] HybridClock initialized: now 1790534795631651 us; error 28 us; skew 500 ppm
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:35.634331 31504 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:35.634795 31504 webserver.cc:460] Webserver started at http://127.15.110.195:41587/ using document root <none> and password file <none>
I20260927 18:46:35.635015 31504 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:35.635080 31504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:35.635248 31504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:46:35.636370 31504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "f4e0622e6e42ca8a2a563ecd82b509e5"
server_key_iv: "362ebd3add13a5233a53338276d13825"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.636724 31504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "f4e0622e6e42ca8a2a563ecd82b509e5"
server_key_iv: "362ebd3add13a5233a53338276d13825"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.638063 31504 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:46:35.638845 31519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.639031 31504 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:46:35.639127 31504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "f4e0622e6e42ca8a2a563ecd82b509e5"
server_key_iv: "362ebd3add13a5233a53338276d13825"
server_key_version: "encryptionkey@0"
I20260927 18:46:35.639228 31504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:35.659780 31504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:35.662200 31504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:35.662375 31504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:35.662724 31504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:35.663137 31504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:35.663210 31504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.663256 31504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:35.663283 31504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:35.669282 31504 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40871
I20260927 18:46:35.669368 31632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40871 every 8 connection(s)
I20260927 18:46:35.669704 31504 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:35.674180 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31504
I20260927 18:46:35.674299 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:46:35.674481 15803 external_mini_cluster.cc:1428] Setting key deca48044468e0a0007c14e7a89f23cf
Sep 27 18:46:35 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534795, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:35.679666 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.670503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:37521 (local address 127.15.110.254:42323)
0927 18:46:35.670545 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.670547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.670771 (+ 224us) server_negotiation.cc:407] Connection header received
0927 18:46:35.671410 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.671411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.671423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.671478 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:35.672446 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.672946 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.673973 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.674162 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.674259 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.676765 (+ 2506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:35.676767 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:35.676767 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:35.676774 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:35.678213 (+ 1439us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.678662 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.678664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.678664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.678721 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.678982 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.678983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.678984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.679099 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:35.679177 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.679491 (+ 314us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.679573 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:46:35.679875 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.670462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:37521)
0927 18:46:35.670630 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.670673 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.671304 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.671553 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.671557 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.672141 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:35.672338 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.672342 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.673038 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.673039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.673866 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.673868 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.674267 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.674268 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.674409 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.675054 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:35.675066 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:35.676643 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:35.678315 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.678329 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.678332 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.678567 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.678798 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:35.678799 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:35.678800 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:35.678912 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:35.679167 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:35.679196 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:35.679354 (+ 158us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.679467 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.679627 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":91,"threads_started":1}
I20260927 18:46:35.680294 31633 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:35.680393 31633 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:35.680532 31633 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:35.680871 31174 ts_manager.cc:194] Registered new tserver with Master: fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871)
I20260927 18:46:35.681193 31174 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:37521
I20260927 18:46:35.686327 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:35.693902 31214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.687674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.15.110.254:42323)
0927 18:46:35.687764 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.687765 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.687843 (+ 78us) server_negotiation.cc:407] Connection header received
0927 18:46:35.687892 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.687892 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.687900 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.687947 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:35.688450 (+ 503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.688945 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.689678 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.689888 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.689963 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.691213 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:35.691215 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:35.691215 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:35.691222 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:35.692712 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.693076 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.693077 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.693078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.693131 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:35.693337 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:35.693338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:35.693338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:35.693524 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:35.693617 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.693725 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.693802 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260927 18:46:35.695869 31174 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56592:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:46:35.697537 31174 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:46:35.704101 31645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40871 (local address 127.0.0.1:58438)
0927 18:46:35.700162 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.700209 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.700279 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.700497 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.700500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.700502 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:35.700568 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.700570 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.701841 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.701842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.702856 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.702858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.703781 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.703782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.703866 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.703880 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.703975 (+ 95us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.704028 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":69,"threads_started":1}
I20260927 18:46:35.704113 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58438 (local address 127.15.110.195:40871)
0927 18:46:35.700265 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.700270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.700282 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:46:35.700370 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.700372 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.700388 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.700491 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:35.700988 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.701708 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.702950 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.703680 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.703781 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.703841 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.703992 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.704034 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":84,"threads_started":1}
I20260927 18:46:35.704227 31647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33281 (local address 127.0.0.1:51172)
0927 18:46:35.700357 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.700369 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.700423 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.700750 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.700751 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.700753 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:35.700809 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.700810 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.702219 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.702220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.703228 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.703231 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.704022 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.704023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.704087 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.704091 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.704120 (+ 29us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.704156 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":92,"threads_started":1}
I20260927 18:46:35.704645 31648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.15.110.194:33281)
0927 18:46:35.700565 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.700570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.700581 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:35.700622 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.700624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.700641 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.700889 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:35.700971 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.702104 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.703306 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.703923 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.704427 (+ 504us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.704489 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.704516 (+ 27us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.704557 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260927 18:46:35.704694 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.0.0.1:53762)
0927 18:46:35.700682 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:35.700691 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:35.700735 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:35.701147 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:35.701148 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:35.701151 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:35.701193 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.701195 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.702456 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.702457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.703534 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:35.703536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.704460 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.704461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:35.704518 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.704522 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:46:35.704575 (+ 53us) client_negotiation.cc:240] Negotiation successful
0927 18:46:35.704622 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":101,"threads_started":1}
I20260927 18:46:35.704716 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:35.700695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53762 (local address 127.15.110.193:44351)
0927 18:46:35.700979 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:35.700983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:35.700993 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:46:35.701034 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:35.701036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:35.701051 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:35.701273 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:35.701341 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.702353 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.703626 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:35.704385 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:35.704451 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:35.704495 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:35.704584 (+ 89us) server_negotiation.cc:299] Negotiation successful
0927 18:46:35.704629 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":120,"threads_started":1}
I20260927 18:46:35.705264 31433 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 (DEFAULT_TABLE table=test-table [id=e68af92a99db42f5acef89d5044d2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:35.705250 31567 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 (DEFAULT_TABLE table=test-table [id=e68af92a99db42f5acef89d5044d2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:35.705391 31299 tablet_service.cc:1467] Processing CreateTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 (DEFAULT_TABLE table=test-table [id=e68af92a99db42f5acef89d5044d2359]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:35.705596 31567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5cdbb69577469f841b82a5cd3d2924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:35.705596 31433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5cdbb69577469f841b82a5cd3d2924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:35.705641 31299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a5cdbb69577469f841b82a5cd3d2924. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:35.708556 31650 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap starting.
I20260927 18:46:35.708848 31652 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Bootstrap starting.
I20260927 18:46:35.708850 31651 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Bootstrap starting.
I20260927 18:46:35.709244 31650 tablet_bootstrap.cc:654] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:35.709486 31652 tablet_bootstrap.cc:654] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:35.709486 31651 tablet_bootstrap.cc:654] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:35.709790 31650 log.cc:824] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:35.710091 31652 log.cc:824] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:35.710103 31651 log.cc:824] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:35.710569 31650 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: No bootstrap required, opened a new log
I20260927 18:46:35.710709 31650 ts_tablet_manager.cc:1397] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:35.710856 31652 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: No bootstrap required, opened a new log
I20260927 18:46:35.710865 31651 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: No bootstrap required, opened a new log
I20260927 18:46:35.710927 31651 ts_tablet_manager.cc:1397] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:35.710927 31652 ts_tablet_manager.cc:1397] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:46:35.712543 31650 raft_consensus.cc:348] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.712728 31650 raft_consensus.cc:374] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:35.712780 31650 raft_consensus.cc:729] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2236d8bd113a49e79c10857b4ee157c7, State: Initialized, Role: FOLLOWER
I20260927 18:46:35.712896 31650 consensus_queue.cc:260] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.712855 31652 raft_consensus.cc:348] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.712865 31651 raft_consensus.cc:348] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.713011 31651 raft_consensus.cc:374] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:35.713011 31652 raft_consensus.cc:374] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:35.713053 31652 raft_consensus.cc:729] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d890352b11a24b1aab0c1ea6d71c928e, State: Initialized, Role: FOLLOWER
I20260927 18:46:35.713053 31651 raft_consensus.cc:729] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd71de13f3b4510aee9cfe810ed97fb, State: Initialized, Role: FOLLOWER
I20260927 18:46:35.713207 31650 ts_tablet_manager.cc:1428] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:46:35.713171 31651 consensus_queue.cc:260] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.713181 31652 consensus_queue.cc:260] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.713330 31365 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
I20260927 18:46:35.713445 31652 ts_tablet_manager.cc:1428] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:46:35.713445 31651 ts_tablet_manager.cc:1428] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:46:35.713572 31499 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
I20260927 18:46:35.713593 31633 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
W20260927 18:46:35.755789 31500 tablet.cc:2369] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:46:35.854190 31366 tablet.cc:2369] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:46:35.920287 31634 tablet.cc:2369] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:35.999467 31657 raft_consensus.cc:482] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:35.999600 31657 raft_consensus.cc:504] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:35.999993 31657 leader_election.cc:288] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281), 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:36.004042 31649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.000274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:46877 (local address 127.15.110.193:44351)
0927 18:46:36.000366 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:36.000368 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:36.000452 (+ 84us) server_negotiation.cc:407] Connection header received
0927 18:46:36.000480 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:36.000481 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:36.000489 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:36.000538 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:36.000696 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.001455 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003030 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.003644 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003734 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:36.003743 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:36.003923 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 18:46:36.003971 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260927 18:46:36.004042 31648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.000230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:38629 (local address 127.15.110.194:33281)
0927 18:46:36.000295 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:36.000297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:36.000308 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:46:36.000396 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:36.000397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:36.000432 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:36.000490 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:36.000699 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.001465 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003030 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.003647 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003729 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:36.003739 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:36.003923 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:46:36.003971 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:46:36.004019 31635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.000183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33281 (local address 127.15.110.195:38629)
0927 18:46:36.000243 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:36.000256 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:36.000300 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:36.000512 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:36.000512 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:36.000532 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:36.000617 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:36.000619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.001547 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.001548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:36.002869 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:36.002872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.003756 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003757 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:36.003827 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:36.003829 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:46:36.003880 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:46:36.003926 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":60}
I20260927 18:46:36.004027 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.000202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.15.110.195:46877)
0927 18:46:36.000354 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:36.000364 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:36.000419 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:36.000549 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:36.000550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:36.000551 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:36.000617 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:36.000619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.001547 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.001548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:36.002891 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:36.002893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.003737 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.003738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:36.003815 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:36.003820 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:36.003906 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:46:36.003938 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":69,"thread_start_us":93,"threads_started":1}
I20260927 18:46:36.004441 31319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5cdbb69577469f841b82a5cd3d2924" candidate_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2236d8bd113a49e79c10857b4ee157c7" is_pre_election: true
I20260927 18:46:36.004432 31453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5cdbb69577469f841b82a5cd3d2924" candidate_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d890352b11a24b1aab0c1ea6d71c928e" is_pre_election: true
I20260927 18:46:36.004592 31319 raft_consensus.cc:2454] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd71de13f3b4510aee9cfe810ed97fb in term 0.
I20260927 18:46:36.004617 31453 raft_consensus.cc:2454] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fbd71de13f3b4510aee9cfe810ed97fb in term 0.
I20260927 18:46:36.004786 31522 leader_election.cc:302] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7, fbd71de13f3b4510aee9cfe810ed97fb; no voters:
I20260927 18:46:36.004928 31657 raft_consensus.cc:2781] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:36.004997 31657 raft_consensus.cc:482] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:36.005024 31657 raft_consensus.cc:3037] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:36.005764 31657 raft_consensus.cc:504] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:36.005900 31657 leader_election.cc:288] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [CANDIDATE]: Term 1 election: Requested vote from peers d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281), 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:36.006094 31319 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5cdbb69577469f841b82a5cd3d2924" candidate_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2236d8bd113a49e79c10857b4ee157c7"
I20260927 18:46:36.006115 31453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a5cdbb69577469f841b82a5cd3d2924" candidate_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d890352b11a24b1aab0c1ea6d71c928e"
I20260927 18:46:36.006201 31453 raft_consensus.cc:3037] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:36.006187 31319 raft_consensus.cc:3037] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:36.007109 31319 raft_consensus.cc:2454] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd71de13f3b4510aee9cfe810ed97fb in term 1.
I20260927 18:46:36.007109 31453 raft_consensus.cc:2454] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fbd71de13f3b4510aee9cfe810ed97fb in term 1.
I20260927 18:46:36.007342 31522 leader_election.cc:302] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7, fbd71de13f3b4510aee9cfe810ed97fb; no voters:
I20260927 18:46:36.007445 31657 raft_consensus.cc:2781] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:36.007606 31657 raft_consensus.cc:686] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 LEADER]: Becoming Leader. State: Replica: fbd71de13f3b4510aee9cfe810ed97fb, State: Running, Role: LEADER
I20260927 18:46:36.007728 31657 consensus_queue.cc:237] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:36.008577 31174 catalog_manager.cc:5495] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195). New cstate: current_term: 1 leader_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } health_report { overall_health: UNKNOWN } } }
I20260927 18:46:36.057010 31646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.054567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58444 (local address 127.15.110.195:40871)
0927 18:46:36.054640 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:36.054642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:36.054651 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:46:36.054700 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:36.054700 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:36.054719 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:36.054800 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:36.054944 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.055434 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.056218 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:36.056413 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:36.056490 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:36.056646 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:36.056752 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:36.056896 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:36.056914 (+ 18us) server_negotiation.cc:299] Negotiation successful
0927 18:46:36.056936 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:46:36.058841 31453 raft_consensus.cc:1261] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Refusing update from remote peer fbd71de13f3b4510aee9cfe810ed97fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:36.058867 31319 raft_consensus.cc:1261] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Refusing update from remote peer fbd71de13f3b4510aee9cfe810ed97fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:36.059222 31657 consensus_queue.cc:1035] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:36.059398 31660 consensus_queue.cc:1035] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:36.061859 31663 mvcc.cc:205] Tried to move back new op lower bound from 7334030524654108672 to 7334030524448358400. Current Snapshot: MvccSnapshot[applied={T|T < 7334030524654108672}]
I20260927 18:46:36.064219 31174 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:36.064316 31174 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56592:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:36.065052 31174 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Sending DeleteTablet for 3 replicas of tablet 5a5cdbb69577469f841b82a5cd3d2924
I20260927 18:46:36.065392 31433 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:36 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:51172
I20260927 18:46:36.065413 31567 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:36 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:58438
I20260927 18:46:36.065429 31299 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:36 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:53762
I20260927 18:46:36.065704 31673 tablet_replica.cc:330] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: stopping tablet replica
I20260927 18:46:36.065704 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31236
I20260927 18:46:36.065814 31673 raft_consensus.cc:2229] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:36.066604 31673 raft_consensus.cc:2258] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:36.067106 31673 ts_tablet_manager.cc:1905] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:36.069679 31673 ts_tablet_manager.cc:1918] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:36.069772 31673 log.cc:1196] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/5a5cdbb69577469f841b82a5cd3d2924
I20260927 18:46:36.070043 31673 ts_tablet_manager.cc:1939] T 5a5cdbb69577469f841b82a5cd3d2924 P d890352b11a24b1aab0c1ea6d71c928e: Deleting consensus metadata
I20260927 18:46:36.070492 31158 catalog_manager.cc:4841] TS d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281): tablet 5a5cdbb69577469f841b82a5cd3d2924 (table test-table [id=e68af92a99db42f5acef89d5044d2359]) successfully deleted
W20260927 18:46:36.075374 31160 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:44351 (error 108)
I20260927 18:46:36.075398 31674 tablet_replica.cc:330] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: stopping tablet replica
I20260927 18:46:36.075536 31674 raft_consensus.cc:2229] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:46:36.075721 31674 raft_consensus.cc:2258] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:36.075953 31644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:36.075766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.0.0.1:53778)
0927 18:46:36.075869 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:36.075915 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260927 18:46:36.075956 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31370
W20260927 18:46:36.076821 31160 catalog_manager.cc:4584] TS 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a5cdbb69577469f841b82a5cd3d2924: Network error: Client connection negotiation failed: client connection to 127.15.110.193:44351: connect: Connection refused (error 111)
I20260927 18:46:36.077975 31674 ts_tablet_manager.cc:1905] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:36.080026 31674 ts_tablet_manager.cc:1918] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:36.080159 31674 log.cc:1196] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/5a5cdbb69577469f841b82a5cd3d2924
I20260927 18:46:36.080487 31674 ts_tablet_manager.cc:1939] T 5a5cdbb69577469f841b82a5cd3d2924 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting consensus metadata
I20260927 18:46:36.081219 31160 catalog_manager.cc:4841] TS fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): tablet 5a5cdbb69577469f841b82a5cd3d2924 (table test-table [id=e68af92a99db42f5acef89d5044d2359]) successfully deleted
I20260927 18:46:36.081677 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31504
I20260927 18:46:36.085768 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31142
I20260927 18:46:56.091084 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--webserver_interface=127.15.110.254
--webserver_port=44427
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:46:56.173686 31677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:56.173898 31677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:56.173920 31677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:56.175065 31677 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:46:56.175105 31677 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:46:56.175118 31677 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:46:56.175130 31677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:46:56.175143 31677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:56.175194 31677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:56.175230 31677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:46:56.175257 31677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:46:56.176949 31677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=44427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:56.177196 31677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:56.177455 31677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:46:56.186690 31677 server_base.cc:1034] running on GCE node
W20260927 18:46:56.186681 31683 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.186681 31684 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.186892 31686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:56.187166 31677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:56.187575 31677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:56.188745 31677 hybrid_clock.cc:648] HybridClock initialized: now 1790534816188715 us; error 47 us; skew 500 ppm
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:56.197561 31677 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:46:56.197984 31677 webserver.cc:460] Webserver started at http://127.15.110.254:44427/ using document root <none> and password file <none>
I20260927 18:46:56.198179 31677 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:56.198231 31677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:56.199601 31677 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:56.200342 31693 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.200551 31677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:56.200645 31677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "a74443fcc1794ad7ac753bc220b9f853"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "1ed24ffa557b4999331f2daa46e35262"
server_key_iv: "b94521313ac72d796f5a60847134bffa"
server_key_version: "encryptionkey@0"
I20260927 18:46:56.201063 31677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:56.210839 31677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:56.212893 31677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:56.213004 31677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:56.217357 31677 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:42323
I20260927 18:46:56.217384 31745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:42323 every 8 connection(s)
I20260927 18:46:56.217808 31677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:46:56.219436 31746 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:46:56.220085 31746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap starting.
I20260927 18:46:56.226343 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31677
I20260927 18:46:56.226641 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:44351
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=33057
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:56.228546 31746 log.cc:824] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:56.230486 31746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:56.230790 31746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap complete.
I20260927 18:46:56.233493 31746 raft_consensus.cc:348] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:56.233852 31746 raft_consensus.cc:729] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Initialized, Role: FOLLOWER
I20260927 18:46:56.234088 31746 consensus_queue.cc:260] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:56.234200 31746 raft_consensus.cc:388] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:46:56.234268 31746 raft_consensus.cc:482] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:46:56.234323 31746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 1 FOLLOWER]: Advancing to term 2
I20260927 18:46:56.235548 31746 raft_consensus.cc:504] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:56.235694 31746 leader_election.cc:302] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a74443fcc1794ad7ac753bc220b9f853; no voters:
I20260927 18:46:56.235924 31746 leader_election.cc:288] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260927 18:46:56.235962 31750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Leader election won for term 2
I20260927 18:46:56.236089 31750 raft_consensus.cc:686] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 LEADER]: Becoming Leader. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Running, Role: LEADER
I20260927 18:46:56.236217 31746 sys_catalog.cc:564] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:46:56.236272 31750 consensus_queue.cc:237] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:46:56.236670 31752 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a74443fcc1794ad7ac753bc220b9f853. Latest consensus state: current_term: 2 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:46:56.236680 31751 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:46:56.236764 31752 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:56.236785 31751 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
I20260927 18:46:56.238319 31766 catalog_manager.cc:1259] Loaded cluster ID: 73eec58f55e74331b6e92a9182058715
I20260927 18:46:56.238370 31766 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: loading cluster ID for follower catalog manager: success
I20260927 18:46:56.239567 31766 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: acquiring CA information for follower catalog manager: success
I20260927 18:46:56.239996 31766 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:46:56.240067 31756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:46:56.240476 31756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:46:56.240563 31756 catalog_manager.cc:1259] Loaded cluster ID: 73eec58f55e74331b6e92a9182058715
I20260927 18:46:56.240617 31756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:46:56.240788 31756 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:46:56.240877 31756 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Loaded TSK: 0
I20260927 18:46:56.250649 31756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Generated new TSK 1
I20260927 18:46:56.250746 31756 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:46:56.310176 31748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:56.310420 31748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:56.310460 31748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:56.311705 31748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:56.311769 31748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:56.311810 31748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:46:56.313277 31748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=33057
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:56.313549 31748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:56.313781 31748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:56.318805 31773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.318879 31774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.318845 31776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:56.319428 31748 server_base.cc:1034] running on GCE node
I20260927 18:46:56.319646 31748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:56.319921 31748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:56.321120 31748 hybrid_clock.cc:648] HybridClock initialized: now 1790534816321109 us; error 33 us; skew 500 ppm
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:56.323870 31748 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:46:56.324405 31748 webserver.cc:460] Webserver started at http://127.15.110.193:33057/ using document root <none> and password file <none>
I20260927 18:46:56.324653 31748 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:56.324740 31748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:56.326128 31748 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:46:56.326824 31783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.327085 31748 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:56.327221 31748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "2236d8bd113a49e79c10857b4ee157c7"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "8ef1fa9985a52b922c40b755a8490f80"
server_key_iv: "63921fd3650fa9e17d9478290d717169"
server_key_version: "encryptionkey@0"
I20260927 18:46:56.327697 31748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:56.343585 31748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:56.346071 31748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:56.346221 31748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:56.346485 31748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:56.347072 31790 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:46:56.349103 31748 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:46:56.349179 31748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:46:56.349218 31748 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:46:56.350116 31748 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:46:56.350162 31748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:56.350210 31790 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap starting.
I20260927 18:46:56.358031 31748 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:44351
I20260927 18:46:56.358167 31897 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:44351 every 8 connection(s)
I20260927 18:46:56.358556 31748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:46:56.361824 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31748
I20260927 18:46:56.362123 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:33281
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=38397
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:56.362324 31790 log.cc:824] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Log is configured to *not* fsync() on all Append() calls
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:56.369624 31790 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:46:56.369949 31790 tablet_bootstrap.cc:492] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap complete.
I20260927 18:46:56.371140 31790 ts_tablet_manager.cc:1397] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Time spent bootstrapping tablet: real 0.021s user 0.002s sys 0.009s
I20260927 18:46:56.372184 31900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.359359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:53011)
0927 18:46:56.359539 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.359581 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.360317 (+ 736us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.360875 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.360880 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.361299 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:56.361635 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.361645 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.363157 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.363158 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.364412 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.364415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.365244 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.365245 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.365403 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.366062 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:56.366073 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:56.367956 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:56.370246 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.370252 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.370254 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.370568 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.370849 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.370850 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.370850 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.370991 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.371564 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:56.371572 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:56.371723 (+ 151us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.371810 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.371980 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260927 18:46:56.372854 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.359461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:53011 (local address 127.15.110.254:42323)
0927 18:46:56.359662 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.359671 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.359710 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:46:56.360565 (+ 855us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.360573 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.360630 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.360785 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:56.361806 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.363013 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.364520 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.365120 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.365236 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.368098 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:56.368102 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:56.368103 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:56.368115 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:56.370148 (+ 2033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.370655 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.370658 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.370658 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.370775 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.371068 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.371068 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.371069 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.371464 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:56.372477 (+ 1013us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.372549 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.372678 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":88,"threads_started":1}
I20260927 18:46:56.372709 31790 raft_consensus.cc:348] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.373032 31790 raft_consensus.cc:729] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2236d8bd113a49e79c10857b4ee157c7, State: Initialized, Role: FOLLOWER
I20260927 18:46:56.373163 31790 consensus_queue.cc:260] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.373457 31898 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:56.373472 31790 ts_tablet_manager.cc:1428] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:46:56.373579 31898 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:56.373819 31898 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:56.374675 31710 ts_manager.cc:194] Registered new tserver with Master: 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:56.376108 31710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:53011
I20260927 18:46:56.377223 31898 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
I20260927 18:46:56.383268 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.375387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.0.0.1:58734)
0927 18:46:56.379299 (+ 3912us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.379341 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.379416 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.379607 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.379610 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.379611 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:56.379674 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.379676 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.380905 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.380906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.381999 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.382002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.382936 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.382937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.383020 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.383034 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.383111 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.383158 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3873,"thread_start_us":92,"threads_started":1}
I20260927 18:46:56.383284 31906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.376435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.15.110.193:44351)
0927 18:46:56.376656 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.376660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.379419 (+ 2759us) server_negotiation.cc:407] Connection header received
0927 18:46:56.379494 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.379496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.379512 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.379596 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:56.379742 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.380808 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.382088 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.382838 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.382929 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.382981 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.383135 (+ 154us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.383204 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260927 18:46:56.384073 31832 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a5cdbb69577469f841b82a5cd3d2924 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:36 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:58734
I20260927 18:46:56.384653 31907 tablet_replica.cc:330] stopping tablet replica
I20260927 18:46:56.384768 31907 raft_consensus.cc:2229] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:56.384841 31907 raft_consensus.cc:2258] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:56.385133 31907 ts_tablet_manager.cc:1905] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:56.387414 31907 ts_tablet_manager.cc:1918] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:56.387470 31907 log.cc:1196] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal/wals/5a5cdbb69577469f841b82a5cd3d2924
I20260927 18:46:56.387697 31907 ts_tablet_manager.cc:1939] T 5a5cdbb69577469f841b82a5cd3d2924 P 2236d8bd113a49e79c10857b4ee157c7: Deleting consensus metadata
I20260927 18:46:56.388273 31696 catalog_manager.cc:4841] TS 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351): tablet 5a5cdbb69577469f841b82a5cd3d2924 (table test-table [id=e68af92a99db42f5acef89d5044d2359]) successfully deleted
W20260927 18:46:56.454562 31903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:56.454836 31903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:56.454877 31903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:56.456084 31903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:56.456153 31903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:56.456199 31903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:46:56.457768 31903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:33281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=38397
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:56.458026 31903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:56.458256 31903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:56.463457 31912 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.463457 31913 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.463478 31915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:56.463897 31903 server_base.cc:1034] running on GCE node
I20260927 18:46:56.464113 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:56.464357 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:56.465499 31903 hybrid_clock.cc:648] HybridClock initialized: now 1790534816465487 us; error 29 us; skew 500 ppm
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:56.468233 31903 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:46:56.468701 31903 webserver.cc:460] Webserver started at http://127.15.110.194:38397/ using document root <none> and password file <none>
I20260927 18:46:56.468951 31903 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:56.469009 31903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:56.470444 31903 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:46:56.471033 31922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.471261 31903 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:56.471349 31903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "d890352b11a24b1aab0c1ea6d71c928e"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "7b597afda578c175b25f170670fc8ea0"
server_key_iv: "f630c0a5539e8f60bf5844f15da3d3e4"
server_key_version: "encryptionkey@0"
I20260927 18:46:56.471683 31903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:56.490653 31903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:56.493099 31903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:56.493261 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:56.493510 31903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:56.493844 31903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:56.493891 31903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.493922 31903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:56.493950 31903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.499976 31903 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:33281
I20260927 18:46:56.500021 32035 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:33281 every 8 connection(s)
I20260927 18:46:56.501406 31903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:56.508225 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 31903
I20260927 18:46:56.508543 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:40871
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=41587
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:46:56.511272 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.502113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:49719 (local address 127.15.110.254:42323)
0927 18:46:56.502177 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.502180 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.502376 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:46:56.502947 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.502948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.502963 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.503023 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:56.504112 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.504614 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.505655 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.505865 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.505951 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.508371 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:56.508372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:56.508372 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:56.508382 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:56.509786 (+ 1404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.510243 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.510245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.510245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.510305 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.510557 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.510558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.510559 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.510678 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:56.510762 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.511039 (+ 277us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.511133 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:46:56.511437 32038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.502081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:49719)
0927 18:46:56.502247 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.502289 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.502834 (+ 545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.503094 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.503098 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.503748 (+ 650us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:56.504005 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.504011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.504745 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.504745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.505560 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.505563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.505959 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.505961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.506098 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.506632 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:56.506640 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:56.508226 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:56.509894 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.509902 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.509904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.510131 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.510376 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.510376 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.510377 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.510486 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.510752 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:56.510759 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:56.510905 (+ 146us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.511019 (+ 114us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.511179 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260927 18:46:56.511934 32036 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:56.512034 32036 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:56.512234 32036 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:56.512611 31710 ts_manager.cc:194] Registered new tserver with Master: d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
I20260927 18:46:56.513012 31710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:49719
W20260927 18:46:56.590777 32039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:46:56.591007 32039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:46:56.591045 32039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:46:56.592216 32039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:46:56.592284 32039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:46:56.592330 32039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:46:56.593818 32039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:40871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=41587
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:46:56.594084 32039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:46:56.594316 32039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:46:56.599421 32045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:46:56.599412 32047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:56.599431 32039 server_base.cc:1034] running on GCE node
W20260927 18:46:56.599411 32044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:46:56.599859 32039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:46:56.600064 32039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:46:56.601198 32039 hybrid_clock.cc:648] HybridClock initialized: now 1790534816601180 us; error 32 us; skew 500 ppm
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:46:56.603614 32039 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:46:56.604204 32039 webserver.cc:460] Webserver started at http://127.15.110.195:41587/ using document root <none> and password file <none>
I20260927 18:46:56.604460 32039 fs_manager.cc:359] Metadata directory not provided
I20260927 18:46:56.604542 32039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:46:56.605860 32039 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:46:56.606434 32054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.606611 32039 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:46:56.606699 32039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "f4e0622e6e42ca8a2a563ecd82b509e5"
server_key_iv: "362ebd3add13a5233a53338276d13825"
server_key_version: "encryptionkey@0"
I20260927 18:46:56.607084 32039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:46:56.621644 32039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:46:56.624385 32039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:46:56.624560 32039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:46:56.624815 32039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:46:56.625140 32039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:46:56.625195 32039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.625236 32039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:46:56.625264 32039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.631045 32039 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40871
I20260927 18:46:56.631093 32167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40871 every 8 connection(s)
I20260927 18:46:56.631450 32039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:46:56.634815 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 32039
Sep 27 18:46:56 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534816, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:46:56.641920 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.635334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34192 (local address 127.15.110.254:42323)
0927 18:46:56.635482 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.635484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.635496 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:46:56.635579 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.635580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.635590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.635656 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:56.636313 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.636904 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.637775 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.637963 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.638014 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.639215 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:56.639217 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:56.639217 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:56.639224 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:56.640651 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.641046 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.641047 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.641047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.641114 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.641318 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.641320 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.641320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.641444 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:56.641628 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.641665 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.641755 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260927 18:46:56.641921 31901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.632193 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:50401 (local address 127.15.110.254:42323)
0927 18:46:56.632242 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.632245 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.632476 (+ 231us) server_negotiation.cc:407] Connection header received
0927 18:46:56.633022 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.633024 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.633037 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.633090 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:56.634105 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.634611 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.636083 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.636333 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.636421 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.638881 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:56.638882 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:56.638883 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:56.638890 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:56.640493 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.640920 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.640921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.640921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.640983 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.641237 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.641238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.641238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.641355 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:56.641430 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.641710 (+ 280us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.641817 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":20}
I20260927 18:46:56.642102 32170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.632176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:50401)
0927 18:46:56.632347 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.632384 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.632912 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.633200 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.633203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.633801 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:46:56.634002 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.634007 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.634724 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.634725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.636004 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.636007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.636448 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.636449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.636566 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.637132 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:46:56.637140 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:46:56.638771 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:46:56.640590 (+ 1819us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.640597 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.640600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.640824 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.641050 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:46:56.641051 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:46:56.641051 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:46:56.641168 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:46:56.641433 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:46:56.641440 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:46:56.641580 (+ 140us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.641689 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.641849 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":103,"threads_started":1}
I20260927 18:46:56.642527 32168 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42323
I20260927 18:46:56.642640 32168 heartbeater.cc:464] Registering TS with master...
I20260927 18:46:56.642846 32168 heartbeater.cc:511] Master 127.15.110.254:42323 requested a full tablet report, sending...
I20260927 18:46:56.643291 31709 ts_manager.cc:194] Registered new tserver with Master: fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871)
I20260927 18:46:56.643808 31709 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:50401
I20260927 18:46:56.654408 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:46:56.657440 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.654794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.15.110.254:42323)
0927 18:46:56.654857 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.654859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.655035 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:46:56.655086 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.655087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.655124 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.655292 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:56.655422 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.655943 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.656693 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.656914 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.656974 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.657187 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:56.657235 (+ 48us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0927 18:46:56.657381 (+ 146us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34202: token expired
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:46:56.657541 32172 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34202: token expired
W20260927 18:46:56.657644 31639 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.15.110.254:42323)
I20260927 18:46:56.663604 32172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.657927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34212 (local address 127.15.110.254:42323)
0927 18:46:56.657981 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.657983 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.657990 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:46:56.658008 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.658009 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.658017 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.658067 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:46:56.658566 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.659018 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.659801 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.659999 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.660080 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.661232 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:46:56.661233 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:46:56.661234 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:46:56.661241 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:46:56.662369 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.662736 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.662737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.662738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.662803 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:46:56.663002 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:46:56.663003 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:46:56.663004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:46:56.663151 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:46:56.663308 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.663411 (+ 103us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.663489 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:46:56.664732 31709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:46:56.666265 31709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:46:56.668474 31832 tablet_service.cc:1467] Processing CreateTablet for tablet 6f84bbac3a9240358bcdcd1bb10a02f0 (DEFAULT_TABLE table=test-table [id=497ccc1962d34457acca6966964565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:56.668687 31832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f84bbac3a9240358bcdcd1bb10a02f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:56.670300 31790 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap starting.
I20260927 18:46:56.670651 31790 tablet_bootstrap.cc:654] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:56.671231 31790 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: No bootstrap required, opened a new log
I20260927 18:46:56.671281 31790 ts_tablet_manager.cc:1397] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260927 18:46:56.671401 31790 raft_consensus.cc:348] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.671468 31790 raft_consensus.cc:374] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:56.671509 31790 raft_consensus.cc:729] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2236d8bd113a49e79c10857b4ee157c7, State: Initialized, Role: FOLLOWER
I20260927 18:46:56.671552 31790 consensus_queue.cc:260] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.671643 31790 ts_tablet_manager.cc:1428] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260927 18:46:56.672447 31905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.668147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40871 (local address 127.0.0.1:45216)
0927 18:46:56.668264 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.668276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.668309 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.669107 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.669107 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.669109 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:56.669145 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.669146 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.670048 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.670049 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.671153 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.671156 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.672237 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.672238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.672294 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.672296 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.672337 (+ 41us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.672371 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":41}
I20260927 18:46:56.672473 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.668375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45216 (local address 127.15.110.195:40871)
0927 18:46:56.668858 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.668863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.668873 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:46:56.668918 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.668920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.668936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.669219 (+ 283us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:56.669322 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.669960 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.671400 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.672143 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.672239 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.672321 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.672348 (+ 27us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.672389 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":82,"threads_started":1}
I20260927 18:46:56.672448 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.668194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33281 (local address 127.0.0.1:56388)
0927 18:46:56.668478 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:56.668488 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:56.668556 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:56.668759 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:56.668760 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:56.668763 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:56.668844 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.668847 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.670076 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.670077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.670990 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:56.670994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.672187 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.672188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:56.672253 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.672258 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:56.672316 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:46:56.672360 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
I20260927 18:46:56.672473 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:56.668299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56388 (local address 127.15.110.194:33281)
0927 18:46:56.668483 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:56.668489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:56.668538 (+ 49us) server_negotiation.cc:407] Connection header received
0927 18:46:56.668645 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:56.668647 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:56.668663 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:56.668754 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:56.668928 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.669974 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.671260 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:56.672020 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:56.672093 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:56.672154 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:56.672350 (+ 196us) server_negotiation.cc:299] Negotiation successful
0927 18:46:56.672389 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":87,"threads_started":1}
I20260927 18:46:56.673220 32102 tablet_service.cc:1467] Processing CreateTablet for tablet 6f84bbac3a9240358bcdcd1bb10a02f0 (DEFAULT_TABLE table=test-table [id=497ccc1962d34457acca6966964565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:56.673635 32102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f84bbac3a9240358bcdcd1bb10a02f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:56.673560 31970 tablet_service.cc:1467] Processing CreateTablet for tablet 6f84bbac3a9240358bcdcd1bb10a02f0 (DEFAULT_TABLE table=test-table [id=497ccc1962d34457acca6966964565c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:46:56.673938 31970 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f84bbac3a9240358bcdcd1bb10a02f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:46:56.676808 32178 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Bootstrap starting.
I20260927 18:46:56.676901 32179 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Bootstrap starting.
I20260927 18:46:56.677470 32178 tablet_bootstrap.cc:654] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:56.677500 32179 tablet_bootstrap.cc:654] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Neither blocks nor log segments found. Creating new log.
I20260927 18:46:56.677973 32179 log.cc:824] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:56.677975 32178 log.cc:824] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Log is configured to *not* fsync() on all Append() calls
I20260927 18:46:56.678696 32179 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: No bootstrap required, opened a new log
I20260927 18:46:56.678706 32178 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: No bootstrap required, opened a new log
I20260927 18:46:56.678779 32179 ts_tablet_manager.cc:1397] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:46:56.678771 32178 ts_tablet_manager.cc:1397] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:46:56.680406 32179 raft_consensus.cc:348] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.680469 32178 raft_consensus.cc:348] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.680593 32179 raft_consensus.cc:374] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:56.680604 32178 raft_consensus.cc:374] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:46:56.680641 32179 raft_consensus.cc:729] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d890352b11a24b1aab0c1ea6d71c928e, State: Initialized, Role: FOLLOWER
I20260927 18:46:56.680663 32178 raft_consensus.cc:729] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fbd71de13f3b4510aee9cfe810ed97fb, State: Initialized, Role: FOLLOWER
I20260927 18:46:56.680768 32178 consensus_queue.cc:260] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.680763 32179 consensus_queue.cc:260] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:56.681066 32179 ts_tablet_manager.cc:1428] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:46:56.681071 32168 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
I20260927 18:46:56.681147 32036 heartbeater.cc:503] Master 127.15.110.254:42323 was elected leader, sending a full tablet report...
I20260927 18:46:56.681360 32178 ts_tablet_manager.cc:1428] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260927 18:46:56.751989 32037 tablet.cc:2369] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:46:56.859529 31899 tablet.cc:2369] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:46:56.882035 32169 tablet.cc:2369] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:46:57.063398 32183 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:46:57.063520 32183 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:57.064049 32183 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:57.067468 32184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.064321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40871 (local address 127.15.110.194:34307)
0927 18:46:57.064460 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:57.064472 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:57.064538 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:57.064707 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:57.064708 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:57.064715 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:57.064800 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:57.064803 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.065632 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.065633 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:57.066468 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:57.066471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.067141 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.067142 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:57.067262 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:57.067267 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:57.067325 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:46:57.067369 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260927 18:46:57.067497 32176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.064408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:34307 (local address 127.15.110.195:40871)
0927 18:46:57.064480 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:57.064482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:57.064549 (+ 67us) server_negotiation.cc:407] Connection header received
0927 18:46:57.064590 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:57.064591 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:57.064601 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:57.064692 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:57.064947 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.065550 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.066573 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.067060 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.067120 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:57.067128 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:57.067361 (+ 233us) server_negotiation.cc:299] Negotiation successful
0927 18:46:57.067419 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:46:57.067593 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.064530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.15.110.194:38655)
0927 18:46:57.064660 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:57.064672 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:46:57.064719 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:46:57.064946 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:46:57.064947 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:46:57.064949 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:46:57.065019 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:57.065021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.065769 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.065770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:57.066654 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:46:57.066656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.067280 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.067281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:46:57.067385 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:57.067390 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:46:57.067458 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:46:57.067497 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260927 18:46:57.067862 32186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.064652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:38655 (local address 127.15.110.193:44351)
0927 18:46:57.064794 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:57.064796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:57.064803 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:46:57.064829 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:57.064830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:57.064841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:57.064908 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:46:57.065109 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.065603 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.066751 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.067184 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.067283 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:57.067292 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:57.067481 (+ 189us) server_negotiation.cc:299] Negotiation successful
0927 18:46:57.067798 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":56,"threads_started":1}
I20260927 18:46:57.068081 32122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f84bbac3a9240358bcdcd1bb10a02f0" candidate_uuid: "d890352b11a24b1aab0c1ea6d71c928e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" is_pre_election: true
I20260927 18:46:57.068125 31852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f84bbac3a9240358bcdcd1bb10a02f0" candidate_uuid: "d890352b11a24b1aab0c1ea6d71c928e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2236d8bd113a49e79c10857b4ee157c7" is_pre_election: true
I20260927 18:46:57.068269 32122 raft_consensus.cc:2454] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d890352b11a24b1aab0c1ea6d71c928e in term 0.
I20260927 18:46:57.068269 31852 raft_consensus.cc:2454] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d890352b11a24b1aab0c1ea6d71c928e in term 0.
I20260927 18:46:57.068518 31925 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7, d890352b11a24b1aab0c1ea6d71c928e; no voters:
I20260927 18:46:57.068655 32183 raft_consensus.cc:2781] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:46:57.068784 32183 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:46:57.068804 32183 raft_consensus.cc:3037] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:57.069502 32183 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:57.069635 32183 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [CANDIDATE]: Term 1 election: Requested vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351)
I20260927 18:46:57.069810 31852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f84bbac3a9240358bcdcd1bb10a02f0" candidate_uuid: "d890352b11a24b1aab0c1ea6d71c928e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2236d8bd113a49e79c10857b4ee157c7"
I20260927 18:46:57.069800 32122 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f84bbac3a9240358bcdcd1bb10a02f0" candidate_uuid: "d890352b11a24b1aab0c1ea6d71c928e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
I20260927 18:46:57.069893 32122 raft_consensus.cc:3037] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:57.069892 31852 raft_consensus.cc:3037] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:46:57.070765 31852 raft_consensus.cc:2454] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d890352b11a24b1aab0c1ea6d71c928e in term 1.
I20260927 18:46:57.070771 32122 raft_consensus.cc:2454] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d890352b11a24b1aab0c1ea6d71c928e in term 1.
I20260927 18:46:57.070974 31925 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7, d890352b11a24b1aab0c1ea6d71c928e; no voters:
I20260927 18:46:57.071077 32183 raft_consensus.cc:2781] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:46:57.071308 32183 raft_consensus.cc:686] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 1 LEADER]: Becoming Leader. State: Replica: d890352b11a24b1aab0c1ea6d71c928e, State: Running, Role: LEADER
I20260927 18:46:57.071432 32183 consensus_queue.cc:237] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:46:57.072309 31709 catalog_manager.cc:5495] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e reported cstate change: term changed from 0 to 1, leader changed from <none> to d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194). New cstate: current_term: 1 leader_uuid: "d890352b11a24b1aab0c1ea6d71c928e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } health_report { overall_health: UNKNOWN } } }
I20260927 18:46:57.113454 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.110481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56396 (local address 127.15.110.194:33281)
0927 18:46:57.110561 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:46:57.110564 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:46:57.110572 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:46:57.110618 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:46:57.110619 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:46:57.110650 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:46:57.110736 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:46:57.110947 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.111676 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.112502 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:46:57.112740 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:46:57.112823 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:46:57.113024 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:46:57.113181 (+ 157us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:46:57.113306 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0927 18:46:57.113327 (+ 21us) server_negotiation.cc:299] Negotiation successful
0927 18:46:57.113350 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:46:57.115520 31852 raft_consensus.cc:1261] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Refusing update from remote peer d890352b11a24b1aab0c1ea6d71c928e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:57.115541 32122 raft_consensus.cc:1261] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Refusing update from remote peer d890352b11a24b1aab0c1ea6d71c928e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:46:57.115919 32183 consensus_queue.cc:1035] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [LEADER]: Connected to new peer: Peer: permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:57.116101 32187 consensus_queue.cc:1035] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:46:57.118633 32192 mvcc.cc:205] Tried to move back new op lower bound from 7334030610902020096 to 7334030610725298176. Current Snapshot: MvccSnapshot[applied={T|T < 7334030610902020096}]
I20260927 18:46:57.118924 32191 mvcc.cc:205] Tried to move back new op lower bound from 7334030610902020096 to 7334030610725298176. Current Snapshot: MvccSnapshot[applied={T|T < 7334030610902020096}]
I20260927 18:46:57.120764 31710 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:57.120847 31710 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:46:57.121575 31710 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Sending DeleteTablet for 3 replicas of tablet 6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:46:57.121922 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31748
I20260927 18:46:57.121915 31970 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f84bbac3a9240358bcdcd1bb10a02f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:57 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:56388
I20260927 18:46:57.122496 32102 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f84bbac3a9240358bcdcd1bb10a02f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:46:57 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:45216
I20260927 18:46:57.122735 32200 tablet_replica.cc:330] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: stopping tablet replica
I20260927 18:46:57.122810 32200 raft_consensus.cc:2229] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:46:57.122880 32200 raft_consensus.cc:2258] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:57.123322 32200 ts_tablet_manager.cc:1905] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:46:57.124953 32200 ts_tablet_manager.cc:1918] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:57.125006 32200 log.cc:1196] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:46:57.125236 32200 ts_tablet_manager.cc:1939] T 6f84bbac3a9240358bcdcd1bb10a02f0 P fbd71de13f3b4510aee9cfe810ed97fb: Deleting consensus metadata
I20260927 18:46:57.125581 31696 catalog_manager.cc:4841] TS fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): tablet 6f84bbac3a9240358bcdcd1bb10a02f0 (table test-table [id=497ccc1962d34457acca6966964565c0]) successfully deleted
I20260927 18:46:57.127355 32199 tablet_replica.cc:330] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: stopping tablet replica
I20260927 18:46:57.127486 32199 raft_consensus.cc:2229] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:46:57.127604 32199 raft_consensus.cc:2258] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:46:57.128050 32199 ts_tablet_manager.cc:1905] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:46:57.129560 31696 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:44351 (error 108)
I20260927 18:46:57.129829 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31903
I20260927 18:46:57.130069 32199 ts_tablet_manager.cc:1918] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:46:57.130132 32199 log.cc:1196] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:46:57.130563 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.129949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:44351 (local address 127.0.0.1:58750)
0927 18:46:57.130487 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:57.130530 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:44351: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":516}
W20260927 18:46:57.130692 31696 catalog_manager.cc:4584] TS 2236d8bd113a49e79c10857b4ee157c7 (127.15.110.193:44351): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f84bbac3a9240358bcdcd1bb10a02f0: Network error: Client connection negotiation failed: client connection to 127.15.110.193:44351: connect: Connection refused (error 111)
I20260927 18:46:57.135227 32174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:46:57.135066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33281 (local address 127.0.0.1:56404)
0927 18:46:57.135142 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:46:57.135176 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.194:33281: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:46:57.135223 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 32039
I20260927 18:46:57.140177 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 31677
I20260927 18:47:17.145480 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--webserver_interface=127.15.110.254
--webserver_port=44427
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42323
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:17.227696 32202 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:17.227898 32202 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:17.227921 32202 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:17.229085 32202 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260927 18:47:17.229125 32202 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260927 18:47:17.229137 32202 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260927 18:47:17.229149 32202 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:17.229162 32202 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:17.229175 32202 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:17.229228 32202 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:17.229244 32202 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:17.230749 32202 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=44427
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32202
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:17.230988 32202 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:17.231454 32202 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:17.237185 32207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.237162 32208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.237195 32202 server_base.cc:1034] running on GCE node
W20260927 18:47:17.237162 32210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.237639 32202 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:17.237948 32202 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:17.239094 32202 hybrid_clock.cc:648] HybridClock initialized: now 1790534837239072 us; error 36 us; skew 500 ppm
Sep 27 18:47:17 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:17.246376 32202 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:17.246814 32202 webserver.cc:460] Webserver started at http://127.15.110.254:44427/ using document root <none> and password file <none>
I20260927 18:47:17.247048 32202 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:17.247120 32202 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:17.248457 32202 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:17.249127 32217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.249293 32202 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:17.249392 32202 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "a74443fcc1794ad7ac753bc220b9f853"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "1ed24ffa557b4999331f2daa46e35262"
server_key_iv: "b94521313ac72d796f5a60847134bffa"
server_key_version: "encryptionkey@0"
I20260927 18:47:17.249748 32202 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:17.262146 32202 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:17.264026 32202 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:17.264182 32202 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:17.268106 32202 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:42323
I20260927 18:47:17.268141 32269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:42323 every 8 connection(s)
I20260927 18:47:17.268522 32202 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:17.269940 32270 sys_catalog.cc:263] Verifying existing consensus state
I20260927 18:47:17.270601 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 32202
I20260927 18:47:17.270689 32270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap starting.
I20260927 18:47:17.270890 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:44351
--local_ip_for_outbound_sockets=127.15.110.193
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=33057
--webserver_interface=127.15.110.193
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:47:17.279459 32270 log.cc:824] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:17.281665 32270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:47:17.281970 32270 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Bootstrap complete.
I20260927 18:47:17.284258 32270 raft_consensus.cc:348] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:47:17.284572 32270 raft_consensus.cc:729] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Initialized, Role: FOLLOWER
I20260927 18:47:17.284775 32270 consensus_queue.cc:260] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:47:17.284873 32270 raft_consensus.cc:388] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:17.284925 32270 raft_consensus.cc:482] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:17.284972 32270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 2 FOLLOWER]: Advancing to term 3
I20260927 18:47:17.285923 32270 raft_consensus.cc:504] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:47:17.286036 32270 leader_election.cc:302] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a74443fcc1794ad7ac753bc220b9f853; no voters:
I20260927 18:47:17.286202 32270 leader_election.cc:288] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260927 18:47:17.286268 32274 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 3 FOLLOWER]: Leader election won for term 3
I20260927 18:47:17.286432 32274 raft_consensus.cc:686] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [term 3 LEADER]: Becoming Leader. State: Replica: a74443fcc1794ad7ac753bc220b9f853, State: Running, Role: LEADER
I20260927 18:47:17.286471 32270 sys_catalog.cc:564] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:17.286597 32274 consensus_queue.cc:237] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } }
I20260927 18:47:17.287039 32276 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a74443fcc1794ad7ac753bc220b9f853. Latest consensus state: current_term: 3 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:47:17.287119 32276 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:17.287351 32275 sys_catalog.cc:455] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a74443fcc1794ad7ac753bc220b9f853" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a74443fcc1794ad7ac753bc220b9f853" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42323 } } }
I20260927 18:47:17.287454 32275 sys_catalog.cc:458] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:17.287603 32280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:17.288394 32280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:17.288668 32280 catalog_manager.cc:1259] Loaded cluster ID: 73eec58f55e74331b6e92a9182058715
I20260927 18:47:17.288743 32280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:17.289955 32280 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:17.290130 32280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Loaded TSK: 0
I20260927 18:47:17.290181 32280 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a74443fcc1794ad7ac753bc220b9f853: Loaded TSK: 1
F20260927 18:47:17.290410 32280 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdb514be9ad google::LogMessage::Fail() at ??:0
@ 0x7fdb514c188a google::LogMessage::SendToLog() at ??:0
@ 0x7fdb514be4de google::LogMessage::Flush() at ??:0
@ 0x7fdb514bfa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdb53c2b104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fdb53c28d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fdb594181cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fdb5941848e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fdb59418594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fdb593dec48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fdb593e8ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdb5338c0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdb53381954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdb541e56db start_thread at ??:0
@ 0x7fdb4ffbc61f clone at ??:0
W20260927 18:47:17.368726 32272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:17.368965 32272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:17.369004 32272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:17.370149 32272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:17.370214 32272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:17.370261 32272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:17.371815 32272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:44351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=33057
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:17.372087 32272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:17.372318 32272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:17.377264 32312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.377334 32311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.377265 32314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.377557 32272 server_base.cc:1034] running on GCE node
I20260927 18:47:17.377777 32272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:17.378021 32272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:17.379225 32272 hybrid_clock.cc:648] HybridClock initialized: now 1790534837379209 us; error 31 us; skew 500 ppm
Sep 27 18:47:17 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:17.381793 32272 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:47:17.382287 32272 webserver.cc:460] Webserver started at http://127.15.110.193:33057/ using document root <none> and password file <none>
I20260927 18:47:17.382521 32272 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:17.382591 32272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:17.384056 32272 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:17.384764 32321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.384944 32272 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:17.385048 32272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "2236d8bd113a49e79c10857b4ee157c7"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "8ef1fa9985a52b922c40b755a8490f80"
server_key_iv: "63921fd3650fa9e17d9478290d717169"
server_key_version: "encryptionkey@0"
I20260927 18:47:17.385406 32272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:17.409415 32272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:17.411680 32272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:17.411829 32272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:17.412081 32272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:17.412567 32328 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260927 18:47:17.414546 32272 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260927 18:47:17.414608 32272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260927 18:47:17.414656 32272 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260927 18:47:17.415354 32272 ts_tablet_manager.cc:610] Registered 1 tablets
I20260927 18:47:17.415416 32272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:17.415483 32328 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap starting.
I20260927 18:47:17.422958 32272 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:44351
I20260927 18:47:17.423027 32435 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:44351 every 8 connection(s)
I20260927 18:47:17.423498 32272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:17.426409 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 32272
I20260927 18:47:17.426522 32328 log.cc:824] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:17.426710 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:33281
--local_ip_for_outbound_sockets=127.15.110.194
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=38397
--webserver_interface=127.15.110.194
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:47:17.429762 32328 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260927 18:47:17.430117 32328 tablet_bootstrap.cc:492] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Bootstrap complete.
I20260927 18:47:17.431046 32328 ts_tablet_manager.cc:1397] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.014s
I20260927 18:47:17.432145 32328 raft_consensus.cc:348] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:17.432400 32328 raft_consensus.cc:729] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2236d8bd113a49e79c10857b4ee157c7, State: Initialized, Role: FOLLOWER
I20260927 18:47:17.432586 32328 consensus_queue.cc:260] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:17.432893 32328 ts_tablet_manager.cc:1428] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260927 18:47:17.519222 32438 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:47:17.424212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:44875)
0927 18:47:17.424394 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.424441 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:17.424895 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:17.518949 (+ 94054us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
W20260927 18:47:17.519527 32436 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:42323 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:42323: Client connection negotiation failed: client connection to 127.15.110.254:42323: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:47:17.519881 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.519759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:50565)
0927 18:47:17.519814 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.519851 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:47:17.520254 32438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.520169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:38477)
0927 18:47:17.520215 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.520239 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260927 18:47:17.520382 32436 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260927 18:47:17.538085 32440 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:17.538317 32440 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:17.538357 32440 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:17.539582 32440 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:17.539641 32440 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:17.539688 32440 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:17.541149 32440 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:33281
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=38397
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32440
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:17.541411 32440 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:17.541649 32440 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:17.546888 32449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.546896 32446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.546906 32447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.547077 32440 server_base.cc:1034] running on GCE node
I20260927 18:47:17.547443 32440 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:17.547688 32440 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:17.548831 32440 hybrid_clock.cc:648] HybridClock initialized: now 1790534837548818 us; error 30 us; skew 500 ppm
Sep 27 18:47:17 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:17.551292 32440 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:47:17.551764 32440 webserver.cc:460] Webserver started at http://127.15.110.194:38397/ using document root <none> and password file <none>
I20260927 18:47:17.552016 32440 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:17.552084 32440 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:17.553416 32440 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:17.554116 32456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.554302 32440 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:17.554397 32440 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "d890352b11a24b1aab0c1ea6d71c928e"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "7b597afda578c175b25f170670fc8ea0"
server_key_iv: "f630c0a5539e8f60bf5844f15da3d3e4"
server_key_version: "encryptionkey@0"
I20260927 18:47:17.554745 32440 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:17.568073 32440 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:17.569880 32440 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:17.570045 32440 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:17.570284 32440 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:17.570696 32463 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260927 18:47:17.572153 32463 ts_tablet_manager.cc:1852] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260927 18:47:17.572428 32463 ts_tablet_manager.cc:1905] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:17.574690 32463 ts_tablet_manager.cc:1918] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:17.574762 32463 log.cc:1196] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:47:17.574970 32463 ts_tablet_manager.cc:1939] T 6f84bbac3a9240358bcdcd1bb10a02f0 P d890352b11a24b1aab0c1ea6d71c928e: Deleting consensus metadata
I20260927 18:47:17.575122 32440 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260927 18:47:17.575201 32440 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260927 18:47:17.575249 32440 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:17.575273 32440 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.581281 32440 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:33281
I20260927 18:47:17.581333 32570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:33281 every 8 connection(s)
I20260927 18:47:17.581626 32440 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:17.582759 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.582269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:39429)
0927 18:47:17.582444 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.582588 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
W20260927 18:47:17.583086 32571 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:42323 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:42323: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
I20260927 18:47:17.583431 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.583334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:56339)
0927 18:47:17.583380 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.583410 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260927 18:47:17.583720 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 32440
I20260927 18:47:17.583796 32573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.583686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:43067)
0927 18:47:17.583731 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.583760 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:47:17.583899 32571 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:47:17.584034 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:40871
--local_ip_for_outbound_sockets=127.15.110.195
--tserver_master_addrs=127.15.110.254:42323
--webserver_port=41587
--webserver_interface=127.15.110.195
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:37777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:17.667554 32574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:17.667789 32574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:17.667815 32574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:17.669204 32574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:17.669253 32574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:17.669282 32574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:17.670912 32574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:37777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:40871
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=41587
--tserver_master_addrs=127.15.110.254:42323
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:17.671202 32574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:17.671460 32574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:17.677078 32580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:17.677078 32582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.677152 32574 server_base.cc:1034] running on GCE node
W20260927 18:47:17.677073 32579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:17.677556 32574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:17.677819 32574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:17.679008 32574 hybrid_clock.cc:648] HybridClock initialized: now 1790534837679006 us; error 42 us; skew 500 ppm
Sep 27 18:47:17 dist-test-slave-vs05 krb5kdc[31129](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:17.681556 32574 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:47:17.681950 32574 webserver.cc:460] Webserver started at http://127.15.110.195:41587/ using document root <none> and password file <none>
I20260927 18:47:17.682130 32574 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:17.682170 32574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:17.683487 32574 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:17.684048 32589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.684229 32574 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:17.684316 32574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "fbd71de13f3b4510aee9cfe810ed97fb"
format_stamp: "Formatted at 2026-09-27 18:46:35 on dist-test-slave-vs05"
server_key: "f4e0622e6e42ca8a2a563ecd82b509e5"
server_key_iv: "362ebd3add13a5233a53338276d13825"
server_key_version: "encryptionkey@0"
I20260927 18:47:17.684670 32574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:17.705274 32574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:17.707700 32574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:17.707870 32574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:17.708117 32574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:17.708429 32574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:17.708462 32574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.708508 32574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:17.708534 32574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:17.714392 32574 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40871
I20260927 18:47:17.714426 32702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40871 every 8 connection(s)
I20260927 18:47:17.714763 32574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:47:17.715849 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.715444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:44655)
0927 18:47:17.715601 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.715672 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
W20260927 18:47:17.716265 32703 heartbeater.cc:644] Failed to heartbeat to 127.15.110.254:42323 (0 consecutive failures): Network error: Failed to ping master at 127.15.110.254:42323: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
I20260927 18:47:17.716566 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.716458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:37703)
0927 18:47:17.716511 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.716536 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 18:47:17.716830 32705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:17.716749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:58643)
0927 18:47:17.716790 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:17.716811 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 18:47:17.716941 32703 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260927 18:47:17.719146 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 32574
I20260927 18:47:18.521024 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.520791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:55725)
0927 18:47:18.520928 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:18.520967 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":59,"threads_started":1}
I20260927 18:47:18.584421 32708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.584221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:47969)
0927 18:47:18.584345 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:18.584383 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":56,"threads_started":1}
I20260927 18:47:18.717587 32709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.717353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:49587)
0927 18:47:18.717468 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:18.717511 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":53,"threads_started":1}
I20260927 18:47:18.995981 32710 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:18.996102 32710 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:18.996510 32710 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
Sep 27 18:47:18 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.194@KRBTEST.COM
Sep 27 18:47:19 dist-test-slave-vs05 krb5kdc[31129](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534837, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.195@KRBTEST.COM
I20260927 18:47:19.007455 32712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.996779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:59485 (local address 127.15.110.195:40871)
0927 18:47:18.996971 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:18.996984 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:18.997024 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:47:18.997469 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:18.997471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:18.997513 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:18.997671 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:18.998451 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:18.999360 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000332 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:19.000725 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000850 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:19.003615 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:19.003619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:19.003619 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:19.003630 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:19.005277 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:19.005773 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:19.005776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:19.005777 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:19.005890 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:19.006251 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:19.006252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:19.006253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:19.006577 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:19.007070 (+ 493us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:19.007141 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:47:19.007334 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":59,"threads_started":1}
I20260927 18:47:19.007462 32707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.996664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40871 (local address 127.15.110.193:59485)
0927 18:47:18.996721 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:18.996736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:18.996799 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:18.997704 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:18.997707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:18.998101 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:18.998302 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:18.998309 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:18.999463 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:18.999464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:19.000239 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:19.000241 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:19.000821 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000822 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:19.000934 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:19.001580 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:19.001584 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:19.003508 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:19.005384 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:19.005385 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:19.005386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:19.005686 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:19.005973 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:19.005974 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:19.005975 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:19.006164 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:19.006667 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:19.006675 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:19.006848 (+ 173us) client_negotiation.cc:769] Sending connection context
0927 18:47:19.006942 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 18:47:19.007317 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":83}
I20260927 18:47:19.007871 32713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.996829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:54027 (local address 127.15.110.194:33281)
0927 18:47:18.997016 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:18.997025 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:18.997042 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:47:18.997502 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:18.997505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:18.997541 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:18.997680 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:18.998462 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:18.999361 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000341 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:19.000725 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000850 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:19.003255 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:19.003259 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:19.003259 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:19.003269 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:19.005155 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:19.005773 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:19.005776 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:19.005776 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:19.005900 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:19.006226 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:19.006227 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:19.006227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:19.006576 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:19.006661 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:19.007630 (+ 969us) server_negotiation.cc:299] Negotiation successful
0927 18:47:19.007770 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":79,"threads_started":1}
I20260927 18:47:19.008235 32711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:18.996725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:33281 (local address 127.15.110.193:54027)
0927 18:47:18.996857 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:18.996868 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:18.996922 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:18.997706 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:18.997708 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:18.998101 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:18.998302 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:18.998308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:18.999490 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:18.999490 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:19.000241 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:19.000244 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:19.000824 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:19.000825 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:19.000934 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:19.001514 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:19.001522 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:19.003054 (+ 1532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:19.005261 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:19.005269 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:19.005271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:19.005670 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:19.006009 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:19.006010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:19.006011 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:19.006147 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:19.006694 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:19.006695 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:19.006848 (+ 153us) client_negotiation.cc:769] Sending connection context
0927 18:47:19.006951 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:47:19.008100 (+ 1149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":63,"thread_start_us":66,"threads_started":1}
W20260927 18:47:19.009512 32322 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
W20260927 18:47:19.013648 32324 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:47:19.013733 32324 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7; no voters: d890352b11a24b1aab0c1ea6d71c928e, fbd71de13f3b4510aee9cfe810ed97fb
I20260927 18:47:19.013921 32710 raft_consensus.cc:2726] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:47:19.521713 32714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:19.521498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:54371)
0927 18:47:19.521628 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:19.521665 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":56,"threads_started":1}
I20260927 18:47:19.585068 32715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:19.584884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:41621)
0927 18:47:19.584996 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:19.585030 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":58,"threads_started":1}
I20260927 18:47:19.718396 32716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:19.718205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:56483)
0927 18:47:19.718326 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:19.718355 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":69,"threads_started":1}
I20260927 18:47:20.522490 32717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:20.522277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:40821)
0927 18:47:20.522404 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:20.522442 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":64,"threads_started":1}
I20260927 18:47:20.585711 32718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:20.585503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:46781)
0927 18:47:20.585627 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:20.585667 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":60,"threads_started":1}
I20260927 18:47:20.719074 32719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:20.718855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:37119)
0927 18:47:20.718999 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:20.719035 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":72,"threads_started":1}
I20260927 18:47:20.817313 32720 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:20.817445 32720 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:20.817662 32720 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
W20260927 18:47:20.818216 32322 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
W20260927 18:47:20.818315 32324 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:47:20.818403 32324 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7; no voters: d890352b11a24b1aab0c1ea6d71c928e, fbd71de13f3b4510aee9cfe810ed97fb
I20260927 18:47:20.818502 32720 raft_consensus.cc:2726] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:47:21.523322 32721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:21.523037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:48121)
0927 18:47:21.523164 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:21.523260 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260927 18:47:21.586488 32722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:21.586262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:50903)
0927 18:47:21.586412 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:21.586448 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":68,"threads_started":1}
I20260927 18:47:21.719697 32723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:21.719513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:56233)
0927 18:47:21.719626 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:21.719657 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":62,"threads_started":1}
I20260927 18:47:22.524082 32724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:22.523851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:46427)
0927 18:47:22.523995 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:22.524034 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":64,"threads_started":1}
I20260927 18:47:22.587306 32725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:22.587060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:37109)
0927 18:47:22.587217 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:22.587263 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":61,"threads_started":1}
I20260927 18:47:22.720363 32726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:22.720151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:44325)
0927 18:47:22.720287 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:22.720324 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":73,"threads_started":1}
I20260927 18:47:22.760198 32727 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:22.760329 32727 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:22.760541 32727 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
W20260927 18:47:22.760934 32324 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
W20260927 18:47:22.761052 32322 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:47:22.761155 32322 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7; no voters: d890352b11a24b1aab0c1ea6d71c928e, fbd71de13f3b4510aee9cfe810ed97fb
I20260927 18:47:22.761243 32727 raft_consensus.cc:2726] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:47:23.524786 32728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:23.524584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:39803)
0927 18:47:23.524706 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:23.524738 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260927 18:47:23.588145 32729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:23.587869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:39487)
0927 18:47:23.588024 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:23.588083 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":81,"threads_started":1}
I20260927 18:47:23.721086 32730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:23.720876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:52789)
0927 18:47:23.721005 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:23.721044 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":72,"threads_started":1}
I20260927 18:47:24.525609 32731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:24.525381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:56271)
0927 18:47:24.525525 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:24.525560 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":78,"threads_started":1}
I20260927 18:47:24.588833 32732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:24.588623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:54743)
0927 18:47:24.588753 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:24.588789 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":59,"threads_started":1}
I20260927 18:47:24.721815 32733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:24.721601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:58043)
0927 18:47:24.721735 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:24.721774 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":62,"threads_started":1}
I20260927 18:47:25.526368 32734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:25.526131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:52241)
0927 18:47:25.526279 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:25.526318 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260927 18:47:25.589514 32735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:25.589274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:56005)
0927 18:47:25.589420 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:25.589465 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260927 18:47:25.722532 32736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:25.722300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:38257)
0927 18:47:25.722449 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:25.722489 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":60,"threads_started":1}
I20260927 18:47:26.527048 32737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:26.526836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:46063)
0927 18:47:26.526963 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:26.527000 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":83,"threads_started":1}
I20260927 18:47:26.590145 32738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:26.589947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:58015)
0927 18:47:26.590068 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:26.590102 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":57,"threads_started":1}
I20260927 18:47:26.723369 32739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:26.723074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:46121)
0927 18:47:26.723276 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:26.723319 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":78,"threads_started":1}
I20260927 18:47:27.338361 32740 raft_consensus.cc:482] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:27.338469 32740 raft_consensus.cc:504] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fbd71de13f3b4510aee9cfe810ed97fb" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40871 } } peers { permanent_uuid: "d890352b11a24b1aab0c1ea6d71c928e" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 33281 } } peers { permanent_uuid: "2236d8bd113a49e79c10857b4ee157c7" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 44351 } }
I20260927 18:47:27.338627 32740 leader_election.cc:288] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871), d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281)
W20260927 18:47:27.339008 32322 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d890352b11a24b1aab0c1ea6d71c928e (127.15.110.194:33281): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
W20260927 18:47:27.339109 32324 leader_election.cc:341] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fbd71de13f3b4510aee9cfe810ed97fb (127.15.110.195:40871): Not found: Tablet not found: 6f84bbac3a9240358bcdcd1bb10a02f0
I20260927 18:47:27.339223 32324 leader_election.cc:302] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2236d8bd113a49e79c10857b4ee157c7; no voters: d890352b11a24b1aab0c1ea6d71c928e, fbd71de13f3b4510aee9cfe810ed97fb
I20260927 18:47:27.339298 32740 raft_consensus.cc:2726] T 6f84bbac3a9240358bcdcd1bb10a02f0 P 2236d8bd113a49e79c10857b4ee157c7 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260927 18:47:27.527781 32741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:27.527559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:58369)
0927 18:47:27.527698 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:27.527734 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260927 18:47:27.590790 32742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:27.590569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:58859)
0927 18:47:27.590712 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:27.590747 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":84,"threads_started":1}
I20260927 18:47:27.724038 32743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:27.723820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:53571)
0927 18:47:27.723948 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:27.723992 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":66,"threads_started":1}
I20260927 18:47:28.528641 32744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:28.528425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:47103)
0927 18:47:28.528554 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:28.528593 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260927 18:47:28.591532 32745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:28.591330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:59621)
0927 18:47:28.591447 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:28.591486 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260927 18:47:28.724779 32746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:28.724562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:33449)
0927 18:47:28.724687 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:28.724732 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":59,"threads_started":1}
I20260927 18:47:29.529429 32747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:29.529208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:50021)
0927 18:47:29.529347 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:29.529384 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
I20260927 18:47:29.592243 32748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:29.592006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:60939)
0927 18:47:29.592147 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:29.592186 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260927 18:47:29.725549 32749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:29.725301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:41097)
0927 18:47:29.725454 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:29.725501 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":75,"threads_started":1}
I20260927 18:47:30.530326 32750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:30.530053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:55623)
0927 18:47:30.530231 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:30.530279 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":86,"threads_started":1}
I20260927 18:47:30.593143 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:30.592923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:39057)
0927 18:47:30.593060 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:30.593099 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":80,"threads_started":1}
I20260927 18:47:30.726605 32752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:30.726305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:32993)
0927 18:47:30.726506 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:30.726555 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":103,"threads_started":1}
I20260927 18:47:31.531113 32753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:31.530888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:53315)
0927 18:47:31.531023 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:31.531062 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":87,"threads_started":1}
I20260927 18:47:31.593773 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:31.593558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:35897)
0927 18:47:31.593698 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:31.593728 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260927 18:47:31.727530 32755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:31.727246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:49689)
0927 18:47:31.727416 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:31.727469 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":74,"threads_started":1}
I20260927 18:47:32.532006 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:32.531769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.193:42741)
0927 18:47:32.531923 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:32.531959 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260927 18:47:32.594576 32757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:32.594321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.194:43043)
0927 18:47:32.594505 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:32.594535 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":104,"threads_started":1}
I20260927 18:47:32.720293 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 32272
I20260927 18:47:32.725224 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 32440
I20260927 18:47:32.728274 32758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:32.728079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42323 (local address 127.15.110.195:58251)
0927 18:47:32.728202 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:32.728234 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.254:42323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":66,"threads_started":1}
I20260927 18:47:32.729723 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 32574
2026-09-27T18:47:32Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58856 ms)
[----------] 30 tests from SecurityITest (232637 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:32 dist-test-slave-vs05 krb5kdc[32761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:32 dist-test-slave-vs05 krb5kdc[32761](info): set up 2 sockets
Sep 27 18:47:32 dist-test-slave-vs05 krb5kdc[32761](info): commencing operation
krb5kdc: starting...
W20260927 18:47:33.775470 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.026s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:33 dist-test-slave-vs05 krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:47:33.893468 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44997
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:44997
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:33.974082 306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:33.974287 306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:33.974308 306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:33.975648 306 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:33.975692 306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:33.975706 306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:33.975719 306 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:33.975731 306 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:33.977182 306 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:44997
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44997
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:33.977432 306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:33.977674 306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:33.982587 311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:33.982596 314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:33.982673 306 server_base.cc:1034] running on GCE node
W20260927 18:47:33.982602 312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:33.983150 306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:33.983495 306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:33.984637 306 hybrid_clock.cc:648] HybridClock initialized: now 1790534853984630 us; error 32 us; skew 500 ppm
Sep 27 18:47:33 dist-test-slave-vs05 krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534853, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:33.987084 306 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:33.987583 306 webserver.cc:460] Webserver started at http://127.15.110.254:46031/ using document root <none> and password file <none>
I20260927 18:47:33.987803 306 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:33.987879 306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:33.988030 306 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:33.989111 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "4756ac84ced24d13b6ccc8286524bf0b"
format_stamp: "Formatted at 2026-09-27 18:47:33 on dist-test-slave-vs05"
server_key: "1575364ece3475980cfcb04c5b27ddec"
server_key_iv: "3c327b0a8e0a905ae20aef09a4f19e75"
server_key_version: "encryptionkey@0"
I20260927 18:47:33.989476 306 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "4756ac84ced24d13b6ccc8286524bf0b"
format_stamp: "Formatted at 2026-09-27 18:47:33 on dist-test-slave-vs05"
server_key: "1575364ece3475980cfcb04c5b27ddec"
server_key_iv: "3c327b0a8e0a905ae20aef09a4f19e75"
server_key_version: "encryptionkey@0"
I20260927 18:47:33.990772 306 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:33.991573 321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:33.991744 306 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:33.991849 306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "4756ac84ced24d13b6ccc8286524bf0b"
format_stamp: "Formatted at 2026-09-27 18:47:33 on dist-test-slave-vs05"
server_key: "1575364ece3475980cfcb04c5b27ddec"
server_key_iv: "3c327b0a8e0a905ae20aef09a4f19e75"
server_key_version: "encryptionkey@0"
I20260927 18:47:33.991938 306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:34.012946 306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:34.015340 306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:34.015506 306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:34.019706 306 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:44997
I20260927 18:47:34.019763 373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:44997 every 8 connection(s)
I20260927 18:47:34.020102 306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:34.021049 374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:34.023483 374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: Bootstrap starting.
I20260927 18:47:34.024065 374 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:34.024364 374 log.cc:824] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:34.025061 374 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: No bootstrap required, opened a new log
I20260927 18:47:34.026675 374 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } }
I20260927 18:47:34.026819 374 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:34.026870 374 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4756ac84ced24d13b6ccc8286524bf0b, State: Initialized, Role: FOLLOWER
I20260927 18:47:34.027015 374 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } }
I20260927 18:47:34.027076 374 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:34.027096 374 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:34.027165 374 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:34.027757 374 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } }
I20260927 18:47:34.027869 374 leader_election.cc:302] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4756ac84ced24d13b6ccc8286524bf0b; no voters:
I20260927 18:47:34.028045 374 leader_election.cc:288] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:34.028084 377 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:34.028280 377 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [term 1 LEADER]: Becoming Leader. State: Replica: 4756ac84ced24d13b6ccc8286524bf0b, State: Running, Role: LEADER
I20260927 18:47:34.028322 374 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:34.028450 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 306
I20260927 18:47:34.028451 377 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } }
I20260927 18:47:34.028560 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:34.028770 15803 external_mini_cluster.cc:1428] Setting key 3f5f1c64e41e5fb226d69a66710df7c6
I20260927 18:47:34.029488 379 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4756ac84ced24d13b6ccc8286524bf0b. Latest consensus state: current_term: 1 leader_uuid: "4756ac84ced24d13b6ccc8286524bf0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } } }
I20260927 18:47:34.029587 379 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:34.030076 387 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:34.029584 378 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4756ac84ced24d13b6ccc8286524bf0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4756ac84ced24d13b6ccc8286524bf0b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44997 } } }
I20260927 18:47:34.030530 378 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:34.030674 387 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:34.032305 387 catalog_manager.cc:1347] Generated new cluster ID: e054a879304d4f779948b060b3785e23
I20260927 18:47:34.032356 387 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:34.040982 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.029815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47656 (local address 127.15.110.254:44997)
0927 18:47:34.030011 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.030022 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.030056 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:47:34.030179 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.030187 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.030254 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.030404 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.031272 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.032611 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.033663 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.034299 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.034413 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.037169 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.037174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.037174 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:34.037187 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.039311 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.039697 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.039700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.039701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.039826 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.040032 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.040033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.040033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.040354 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.040527 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.040598 (+ 71us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.040745 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":89,"threads_started":1}
I20260927 18:47:34.053212 387 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:34.053930 387 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:34.058718 387 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: Generated new TSK 0
I20260927 18:47:34.058907 387 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:47:34.089527 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:34.169150 400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:34.169356 400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:34.169379 400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:34.170516 400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:34.170555 400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:34.170606 400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:34.172122 400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:34.172377 400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:34.172609 400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:34.177683 408 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.177675 406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.177716 400 server_base.cc:1034] running on GCE node
W20260927 18:47:34.177707 405 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.178200 400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:34.178448 400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:34.179625 400 hybrid_clock.cc:648] HybridClock initialized: now 1790534854179611 us; error 29 us; skew 500 ppm
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:34.181979 400 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:47:34.182407 400 webserver.cc:460] Webserver started at http://127.15.110.193:37517/ using document root <none> and password file <none>
I20260927 18:47:34.182649 400 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:34.182717 400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:34.182870 400 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:34.183980 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "42f51db10d0542f4958eef580e92d704"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6373a7ebd76720550efe6ecf6b3d76e4"
server_key_iv: "5ef8f65881adcc0f76fed8f2294f38ba"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.184327 400 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "42f51db10d0542f4958eef580e92d704"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6373a7ebd76720550efe6ecf6b3d76e4"
server_key_iv: "5ef8f65881adcc0f76fed8f2294f38ba"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.185619 400 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:34.186371 415 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.186537 400 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:34.186638 400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "42f51db10d0542f4958eef580e92d704"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6373a7ebd76720550efe6ecf6b3d76e4"
server_key_iv: "5ef8f65881adcc0f76fed8f2294f38ba"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.186712 400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:34.226485 400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:34.228616 400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:34.228777 400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:34.229053 400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:34.229418 400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:34.229465 400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.229498 400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:34.229530 400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.235545 400 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:33493
I20260927 18:47:34.235595 528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:33493 every 8 connection(s)
I20260927 18:47:34.235908 400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:34.244589 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.236583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:43145 (local address 127.15.110.254:44997)
0927 18:47:34.236625 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.236628 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.236883 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:47:34.237333 (+ 450us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.237334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.237346 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.237405 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.238349 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.238841 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.239885 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.240075 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.240127 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.242121 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.242122 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.242122 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:34.242129 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.243389 (+ 1260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.243733 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.243734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.243735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.243784 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.243994 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.243995 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.243995 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.244092 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.244141 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.244376 (+ 235us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.244480 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:47:34.244591 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 400
I20260927 18:47:34.244619 531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.236561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44997 (local address 127.15.110.193:43145)
0927 18:47:34.236727 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.236769 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.237252 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.237487 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.237490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.238080 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:34.238271 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.238276 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.238926 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.238926 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.239807 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.239809 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.240148 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.240149 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.240245 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.240695 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:34.240704 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:34.242033 (+ 1329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:34.243454 (+ 1421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.243460 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.243461 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.243672 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.243842 (+ 170us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.243843 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.243843 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.243941 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.244146 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:34.244150 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:34.244258 (+ 108us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.244350 (+ 92us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.244464 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260927 18:47:34.244690 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:34.244910 15803 external_mini_cluster.cc:1428] Setting key 49598dc1fd4d0a7f24d444e541175cce
I20260927 18:47:34.245428 529 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44997
I20260927 18:47:34.245528 529 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:34.245705 529 heartbeater.cc:511] Master 127.15.110.254:44997 requested a full tablet report, sending...
I20260927 18:47:34.246286 337 ts_manager.cc:194] Registered new tserver with Master: 42f51db10d0542f4958eef580e92d704 (127.15.110.193:33493)
I20260927 18:47:34.246909 337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:43145
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:47:34.269963 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:34.349764 534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:34.349972 534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:34.349994 534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:34.351210 534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:34.351251 534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:34.351279 534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:34.352699 534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:34.352936 534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:34.353158 534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:34.358234 540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.358237 542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.358234 539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.358565 534 server_base.cc:1034] running on GCE node
I20260927 18:47:34.358795 534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:34.359045 534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:34.360210 534 hybrid_clock.cc:648] HybridClock initialized: now 1790534854360194 us; error 28 us; skew 500 ppm
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:34.362731 534 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:47:34.363279 534 webserver.cc:460] Webserver started at http://127.15.110.194:45417/ using document root <none> and password file <none>
I20260927 18:47:34.363511 534 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:34.363579 534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:34.363737 534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:34.364781 534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "94dadf88736f43eaad86b851968bccac"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "3ef08d79fcbe9a41b75cb0eb29ef9808"
server_key_iv: "ecbb8620c602dfd7ddbbda1f4a802f56"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.365151 534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "94dadf88736f43eaad86b851968bccac"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "3ef08d79fcbe9a41b75cb0eb29ef9808"
server_key_iv: "ecbb8620c602dfd7ddbbda1f4a802f56"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.366474 534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:34.367224 549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.367408 534 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:34.367511 534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "94dadf88736f43eaad86b851968bccac"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "3ef08d79fcbe9a41b75cb0eb29ef9808"
server_key_iv: "ecbb8620c602dfd7ddbbda1f4a802f56"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.367597 534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:34.379376 534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:34.381601 534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:34.381755 534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:34.381995 534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:34.382301 534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:34.382355 534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.382396 534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:34.382424 534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.388513 534 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:41105
I20260927 18:47:34.388563 662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:41105 every 8 connection(s)
I20260927 18:47:34.388893 534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:34.394644 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 534
I20260927 18:47:34.394771 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:34.394970 15803 external_mini_cluster.cc:1428] Setting key 14daa753d694b06b9d769ac103c5b222
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:34.399338 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.389637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:43761 (local address 127.15.110.254:44997)
0927 18:47:34.389719 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.389721 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.389933 (+ 212us) server_negotiation.cc:407] Connection header received
0927 18:47:34.390410 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.390411 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.390423 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.390497 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.391493 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.392008 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.393009 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.393221 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.393300 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.395920 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.395922 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.395922 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:34.395930 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.397472 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.398291 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.398292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.398292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.398363 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.398617 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.398618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.398619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.398766 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.398828 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.399079 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.399171 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:47:34.399420 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.389615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44997 (local address 127.15.110.194:43761)
0927 18:47:34.389814 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.389851 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.390312 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.390594 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.390604 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.391190 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:34.391391 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.391396 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.392105 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.392106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.392919 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.392921 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.393362 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.393363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.393472 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.394011 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:34.394020 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:34.395836 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:34.397840 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.397848 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.397851 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.398190 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.398430 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.398430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.398431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.398543 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.398828 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:34.398834 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:34.398954 (+ 120us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.399036 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.399156 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260927 18:47:34.399886 663 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44997
I20260927 18:47:34.399996 663 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:34.400180 663 heartbeater.cc:511] Master 127.15.110.254:44997 requested a full tablet report, sending...
I20260927 18:47:34.400591 337 ts_manager.cc:194] Registered new tserver with Master: 94dadf88736f43eaad86b851968bccac (127.15.110.194:41105)
I20260927 18:47:34.401145 337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:43761
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:47:34.422884 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:41483
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:34.504549 668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:34.504787 668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:34.504827 668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:34.505968 668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:34.506031 668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:34.506076 668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:34.507570 668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:41483
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44997
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:34.507830 668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:34.508059 668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:34.513119 673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.513128 676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.513145 668 server_base.cc:1034] running on GCE node
W20260927 18:47:34.513127 674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.513533 668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:34.513723 668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:34.514887 668 hybrid_clock.cc:648] HybridClock initialized: now 1790534854514885 us; error 37 us; skew 500 ppm
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:34.517499 668 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:47:34.517940 668 webserver.cc:460] Webserver started at http://127.15.110.195:33057/ using document root <none> and password file <none>
I20260927 18:47:34.518162 668 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:34.518229 668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:34.518383 668 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:34.519471 668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "f43a5b8f06134760be33680665a52d1a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6a6cec311fbfc9a231d02cdb30ba4c29"
server_key_iv: "d4b528d0e708f094ce8ba50220ad7353"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.519824 668 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "f43a5b8f06134760be33680665a52d1a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6a6cec311fbfc9a231d02cdb30ba4c29"
server_key_iv: "d4b528d0e708f094ce8ba50220ad7353"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.521135 668 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:34.521850 683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.522038 668 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:34.522142 668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "f43a5b8f06134760be33680665a52d1a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "6a6cec311fbfc9a231d02cdb30ba4c29"
server_key_iv: "d4b528d0e708f094ce8ba50220ad7353"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.522214 668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:34.532889 668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:34.535130 668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:34.535322 668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:34.535547 668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:34.535849 668 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:34.535903 668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.535944 668 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:34.535974 668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.541906 668 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:44729
I20260927 18:47:34.541950 796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:44729 every 8 connection(s)
I20260927 18:47:34.542237 668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:47:34.547618 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 668
I20260927 18:47:34.547694 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
Sep 27 18:47:34 dist-test-slave-vs05 krb5kdc[32761](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534854, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:34.547917 15803 external_mini_cluster.cc:1428] Setting key 4046c61b3595e3881bfa06f11a906603
I20260927 18:47:34.551088 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.542914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:49195 (local address 127.15.110.254:44997)
0927 18:47:34.542991 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.542993 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.543128 (+ 135us) server_negotiation.cc:407] Connection header received
0927 18:47:34.543651 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.543652 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.543661 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.543709 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.544570 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.545062 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.545988 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.546162 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.546211 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.548373 (+ 2162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.548374 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.548375 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:34.548380 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.549752 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.550159 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.550161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.550161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.550221 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.550474 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.550474 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.550475 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.550585 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.550661 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.550923 (+ 262us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.550998 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260927 18:47:34.551306 799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.542871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44997 (local address 127.15.110.195:49195)
0927 18:47:34.543014 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.543047 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.543565 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.543767 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.543771 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.544305 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:34.544486 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.544490 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.545150 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.545150 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.545917 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.545918 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.546229 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.546229 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.546322 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.546732 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:34.546740 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:34.548229 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:34.549861 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.549867 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.549870 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.550082 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.550288 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:34.550288 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:34.550289 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:34.550400 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:34.550651 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:34.550657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:34.550792 (+ 135us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.550901 (+ 109us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.551053 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":85,"threads_started":1}
I20260927 18:47:34.551756 797 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44997
I20260927 18:47:34.551829 797 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:34.552022 797 heartbeater.cc:511] Master 127.15.110.254:44997 requested a full tablet report, sending...
I20260927 18:47:34.552366 337 ts_manager.cc:194] Registered new tserver with Master: f43a5b8f06134760be33680665a52d1a (127.15.110.195:44729)
I20260927 18:47:34.552758 337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:49195
I20260927 18:47:34.559558 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:34.566391 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.560880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47660 (local address 127.15.110.254:44997)
0927 18:47:34.560942 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.560944 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.561071 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:47:34.561109 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.561109 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.561118 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.561178 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.561691 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.562165 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.562845 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.563016 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.563064 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.564185 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.564186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.564186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:34.564192 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.565518 (+ 1326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.565798 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.565799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.565799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.565844 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:34.565986 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:34.565987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:34.565987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:34.566105 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.566169 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.566251 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.566303 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:47:34.567909 337 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47660:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:34.569443 337 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:34.575965 811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.571904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:44729 (local address 127.0.0.1:38228)
0927 18:47:34.572072 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.572080 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.572113 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.572673 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.572674 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.572675 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:34.572713 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.572714 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573897 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.573898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.574913 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.574915 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.575653 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.575733 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.575746 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.575840 (+ 94us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.575893 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
I20260927 18:47:34.575978 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.572344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.15.110.195:44729)
0927 18:47:34.572526 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.572529 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.572539 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:34.572574 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.572576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.572589 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.572664 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:34.572771 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573798 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.574993 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.575552 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575649 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.575706 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.575862 (+ 156us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.575902 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":81,"threads_started":1}
I20260927 18:47:34.576320 809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.571721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:33493 (local address 127.0.0.1:53698)
0927 18:47:34.571858 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.571870 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.571921 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.572475 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.572476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.572478 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:34.572534 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.572535 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573464 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.573466 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.574624 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.574627 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.576067 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.576069 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.576134 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.576138 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.576190 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.576230 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":76,"threads_started":1}
I20260927 18:47:34.576329 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.571694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:41105 (local address 127.0.0.1:41642)
0927 18:47:34.571816 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.571854 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.571913 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.572390 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.572393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.572396 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:34.572461 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.572463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573866 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.573868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.574893 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.574896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.575902 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.575970 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.575973 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.576025 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.576076 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":64,"threads_started":1}
I20260927 18:47:34.576555 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.572072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53698 (local address 127.15.110.193:33493)
0927 18:47:34.572222 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.572227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.572238 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:47:34.572277 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.572279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.572294 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.572393 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:34.572612 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573362 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.574708 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.575596 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575678 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.575729 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.576440 (+ 711us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.576485 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
I20260927 18:47:34.576719 731 tablet_service.cc:1467] Processing CreateTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a (DEFAULT_TABLE table=test-table [id=f9d9029631044a6ba40dc798301bd65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:34.576978 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.571890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.15.110.194:41105)
0927 18:47:34.572199 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.572203 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.572211 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:47:34.572242 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.572244 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.572256 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.572546 (+ 290us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:34.572621 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.573729 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575001 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.575813 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.575902 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.575948 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.576822 (+ 874us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.576868 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":88,"threads_started":1}
I20260927 18:47:34.577103 731 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d5e39e42b0431591dd7cc72e7e303a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:34.577607 597 tablet_service.cc:1467] Processing CreateTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a (DEFAULT_TABLE table=test-table [id=f9d9029631044a6ba40dc798301bd65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:34.577917 597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d5e39e42b0431591dd7cc72e7e303a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:34.579844 463 tablet_service.cc:1467] Processing CreateTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a (DEFAULT_TABLE table=test-table [id=f9d9029631044a6ba40dc798301bd65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:34.580137 463 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50d5e39e42b0431591dd7cc72e7e303a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:34.580472 814 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Bootstrap starting.
I20260927 18:47:34.581154 814 tablet_bootstrap.cc:654] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:34.581782 814 log.cc:824] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:34.582535 814 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: No bootstrap required, opened a new log
I20260927 18:47:34.582609 814 ts_tablet_manager.cc:1397] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:47:34.583319 815 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Bootstrap starting.
I20260927 18:47:34.583915 815 tablet_bootstrap.cc:654] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:34.584380 815 log.cc:824] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:34.584448 814 raft_consensus.cc:348] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.584664 814 raft_consensus.cc:374] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:34.584709 814 raft_consensus.cc:729] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43a5b8f06134760be33680665a52d1a, State: Initialized, Role: FOLLOWER
I20260927 18:47:34.584836 814 consensus_queue.cc:260] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.585184 815 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: No bootstrap required, opened a new log
I20260927 18:47:34.585196 797 heartbeater.cc:503] Master 127.15.110.254:44997 was elected leader, sending a full tablet report...
I20260927 18:47:34.585258 815 ts_tablet_manager.cc:1397] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:34.585371 817 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Bootstrap starting.
I20260927 18:47:34.585891 817 tablet_bootstrap.cc:654] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:34.585187 814 ts_tablet_manager.cc:1428] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:34.586365 817 log.cc:824] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:34.587006 817 tablet_bootstrap.cc:492] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: No bootstrap required, opened a new log
I20260927 18:47:34.587086 817 ts_tablet_manager.cc:1397] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:34.587054 815 raft_consensus.cc:348] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.587282 815 raft_consensus.cc:374] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:34.587337 815 raft_consensus.cc:729] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94dadf88736f43eaad86b851968bccac, State: Initialized, Role: FOLLOWER
I20260927 18:47:34.587459 815 consensus_queue.cc:260] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.587734 815 ts_tablet_manager.cc:1428] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:47:34.587790 663 heartbeater.cc:503] Master 127.15.110.254:44997 was elected leader, sending a full tablet report...
I20260927 18:47:34.588984 817 raft_consensus.cc:348] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.589141 817 raft_consensus.cc:374] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:34.589174 817 raft_consensus.cc:729] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42f51db10d0542f4958eef580e92d704, State: Initialized, Role: FOLLOWER
I20260927 18:47:34.589286 817 consensus_queue.cc:260] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.589500 817 ts_tablet_manager.cc:1428] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260927 18:47:34.589560 529 heartbeater.cc:503] Master 127.15.110.254:44997 was elected leader, sending a full tablet report...
W20260927 18:47:34.639560 664 tablet.cc:2369] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:47:34.736660 530 tablet.cc:2369] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:34.746052 821 raft_consensus.cc:482] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:34.746235 821 raft_consensus.cc:504] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.746656 821 leader_election.cc:288] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f43a5b8f06134760be33680665a52d1a (127.15.110.195:44729), 42f51db10d0542f4958eef580e92d704 (127.15.110.193:33493)
I20260927 18:47:34.749712 812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.746954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:53779 (local address 127.15.110.193:33493)
0927 18:47:34.747018 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.747020 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.747033 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:47:34.747065 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.747066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.747075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.747148 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:34.747350 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.747905 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.748911 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.749330 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.749427 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.749437 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.749592 (+ 155us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.749629 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:47:34.749738 665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.746835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:33493 (local address 127.15.110.194:53779)
0927 18:47:34.746908 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.746922 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.746970 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.747208 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.747209 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.747211 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:34.747278 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.747280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.748012 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.748013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.748815 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.748817 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.749422 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.749423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.749504 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.749508 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.749582 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.749625 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":32}
I20260927 18:47:34.749749 813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.746957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:44509 (local address 127.15.110.195:44729)
0927 18:47:34.747006 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.747008 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.747308 (+ 300us) server_negotiation.cc:407] Connection header received
0927 18:47:34.747381 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.747383 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.747392 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.747463 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:34.747625 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.748090 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.749003 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.749391 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.749477 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.749486 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.749633 (+ 147us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.749679 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:47:34.749838 823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.746852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:44729 (local address 127.15.110.194:44509)
0927 18:47:34.747230 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.747239 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:34.747313 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:34.747478 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:34.747479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:34.747481 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:34.747553 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.747554 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.748175 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.748176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.748907 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:34.748909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.749477 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.749478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:34.749554 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.749557 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:34.749627 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:47:34.749665 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":169,"thread_start_us":125,"threads_started":1}
I20260927 18:47:34.750156 483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d5e39e42b0431591dd7cc72e7e303a" candidate_uuid: "94dadf88736f43eaad86b851968bccac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42f51db10d0542f4958eef580e92d704" is_pre_election: true
I20260927 18:47:34.750196 751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d5e39e42b0431591dd7cc72e7e303a" candidate_uuid: "94dadf88736f43eaad86b851968bccac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43a5b8f06134760be33680665a52d1a" is_pre_election: true
I20260927 18:47:34.750331 751 raft_consensus.cc:2454] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94dadf88736f43eaad86b851968bccac in term 0.
I20260927 18:47:34.750327 483 raft_consensus.cc:2454] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 94dadf88736f43eaad86b851968bccac in term 0.
I20260927 18:47:34.750573 550 leader_election.cc:302] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94dadf88736f43eaad86b851968bccac, f43a5b8f06134760be33680665a52d1a; no voters:
I20260927 18:47:34.750705 821 raft_consensus.cc:2781] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:34.750761 821 raft_consensus.cc:482] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:34.750784 821 raft_consensus.cc:3037] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:34.751490 821 raft_consensus.cc:504] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.751637 821 leader_election.cc:288] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [CANDIDATE]: Term 1 election: Requested vote from peers f43a5b8f06134760be33680665a52d1a (127.15.110.195:44729), 42f51db10d0542f4958eef580e92d704 (127.15.110.193:33493)
I20260927 18:47:34.751820 751 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d5e39e42b0431591dd7cc72e7e303a" candidate_uuid: "94dadf88736f43eaad86b851968bccac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43a5b8f06134760be33680665a52d1a"
I20260927 18:47:34.751857 483 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50d5e39e42b0431591dd7cc72e7e303a" candidate_uuid: "94dadf88736f43eaad86b851968bccac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "42f51db10d0542f4958eef580e92d704"
I20260927 18:47:34.751915 751 raft_consensus.cc:3037] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:34.751950 483 raft_consensus.cc:3037] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:34.752748 751 raft_consensus.cc:2454] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94dadf88736f43eaad86b851968bccac in term 1.
I20260927 18:47:34.752758 483 raft_consensus.cc:2454] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 94dadf88736f43eaad86b851968bccac in term 1.
I20260927 18:47:34.752919 550 leader_election.cc:302] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 94dadf88736f43eaad86b851968bccac, f43a5b8f06134760be33680665a52d1a; no voters:
I20260927 18:47:34.753032 821 raft_consensus.cc:2781] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:34.753197 821 raft_consensus.cc:686] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 1 LEADER]: Becoming Leader. State: Replica: 94dadf88736f43eaad86b851968bccac, State: Running, Role: LEADER
I20260927 18:47:34.753306 821 consensus_queue.cc:237] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } }
I20260927 18:47:34.754122 337 catalog_manager.cc:5495] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac reported cstate change: term changed from 0 to 1, leader changed from <none> to 94dadf88736f43eaad86b851968bccac (127.15.110.194). New cstate: current_term: 1 leader_uuid: "94dadf88736f43eaad86b851968bccac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94dadf88736f43eaad86b851968bccac" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 41105 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 } health_report { overall_health: UNKNOWN } } }
W20260927 18:47:34.792829 798 tablet.cc:2369] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:34.801751 810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.799194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.15.110.194:41105)
0927 18:47:34.799257 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.799260 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.799269 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:47:34.799353 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.799359 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.799379 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.799481 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:34.799618 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.800132 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.800937 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.801136 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.801211 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.801391 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:34.801496 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:34.801591 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.801610 (+ 19us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.801630 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260927 18:47:34.803679 751 raft_consensus.cc:1261] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 1 FOLLOWER]: Refusing update from remote peer 94dadf88736f43eaad86b851968bccac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:34.803752 483 raft_consensus.cc:1261] T 50d5e39e42b0431591dd7cc72e7e303a P 42f51db10d0542f4958eef580e92d704 [term 1 FOLLOWER]: Refusing update from remote peer 94dadf88736f43eaad86b851968bccac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:34.804046 824 consensus_queue.cc:1035] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [LEADER]: Connected to new peer: Peer: permanent_uuid: "42f51db10d0542f4958eef580e92d704" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 33493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:34.804183 821 consensus_queue.cc:1035] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [LEADER]: Connected to new peer: Peer: permanent_uuid: "f43a5b8f06134760be33680665a52d1a" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44729 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:34.806370 827 mvcc.cc:205] Tried to move back new op lower bound from 7334030765272965120 to 7334030765069991936. Current Snapshot: MvccSnapshot[applied={T|T < 7334030765272965120}]
I20260927 18:47:34.806607 829 mvcc.cc:205] Tried to move back new op lower bound from 7334030765272965120 to 7334030765069991936. Current Snapshot: MvccSnapshot[applied={T|T < 7334030765272965120}]
I20260927 18:47:34.808892 337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:34.808988 337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47660:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:34.809793 337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4756ac84ced24d13b6ccc8286524bf0b: Sending DeleteTablet for 3 replicas of tablet 50d5e39e42b0431591dd7cc72e7e303a
I20260927 18:47:34.810092 597 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:34 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:41642
I20260927 18:47:34.810137 731 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:34 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:38228
I20260927 18:47:34.810163 463 tablet_service.cc:1514] Processing DeleteTablet for tablet 50d5e39e42b0431591dd7cc72e7e303a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:34 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:53698
I20260927 18:47:34.810379 836 tablet_replica.cc:330] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac: stopping tablet replica
I20260927 18:47:34.810495 836 raft_consensus.cc:2229] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:47:34.810655 836 raft_consensus.cc:2258] T 50d5e39e42b0431591dd7cc72e7e303a P 94dadf88736f43eaad86b851968bccac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:34.810732 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 400
I20260927 18:47:34.812204 837 tablet_replica.cc:330] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: stopping tablet replica
I20260927 18:47:34.812321 837 raft_consensus.cc:2229] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:34.812434 837 raft_consensus.cc:2258] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:34.812898 837 ts_tablet_manager.cc:1905] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:34.814796 837 ts_tablet_manager.cc:1918] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:34.814883 837 log.cc:1196] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/50d5e39e42b0431591dd7cc72e7e303a
I20260927 18:47:34.815245 837 ts_tablet_manager.cc:1939] T 50d5e39e42b0431591dd7cc72e7e303a P f43a5b8f06134760be33680665a52d1a: Deleting consensus metadata
I20260927 18:47:34.815691 322 catalog_manager.cc:4841] TS f43a5b8f06134760be33680665a52d1a (127.15.110.195:44729): tablet 50d5e39e42b0431591dd7cc72e7e303a (table test-table [id=f9d9029631044a6ba40dc798301bd65c]) successfully deleted
W20260927 18:47:34.816672 325 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:33493 (error 108)
I20260927 18:47:34.816882 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 534
I20260927 18:47:34.817481 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.817089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:33493 (local address 127.0.0.1:53710)
0927 18:47:34.817408 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.817452 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:33493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":297}
W20260927 18:47:34.817602 325 catalog_manager.cc:4584] TS 42f51db10d0542f4958eef580e92d704 (127.15.110.193:33493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50d5e39e42b0431591dd7cc72e7e303a: Network error: Client connection negotiation failed: client connection to 127.15.110.193:33493: connect: Connection refused (error 111)
I20260927 18:47:34.822166 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 668
I20260927 18:47:34.822224 808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.822094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:41105 (local address 127.0.0.1:41662)
0927 18:47:34.822134 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:34.822186 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.194:41105: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260927 18:47:34.826561 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 306
2026-09-27T18:47:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2097 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-27T18:47:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:34Z Disabled control of system clock
I20260927 18:47:34.854393 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44641
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:44641
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:34.935702 843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:34.935930 843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:34.935951 843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:34.937079 843 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:34.937119 843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:34.937132 843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:34.937145 843 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:34.937157 843 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:34.938519 843 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:44641
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:34.938719 843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:34.938964 843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:34.943661 851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.943679 849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:34.943679 848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:34.944042 843 server_base.cc:1034] running on GCE node
I20260927 18:47:34.944293 843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:34.944567 843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:34.945732 843 hybrid_clock.cc:648] HybridClock initialized: now 1790534854945717 us; error 26 us; skew 500 ppm
I20260927 18:47:34.947024 843 webserver.cc:460] Webserver started at http://127.15.110.254:46397/ using document root <none> and password file <none>
I20260927 18:47:34.947324 843 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:34.947407 843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:34.947568 843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:34.948669 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "1c32d3c107854ef8a59239015704099a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "f751287d681e11a9093287118cbaf271"
server_key_iv: "2b334853cefcfc5b7a7b6f08cf40b8cd"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.949033 843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "1c32d3c107854ef8a59239015704099a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "f751287d681e11a9093287118cbaf271"
server_key_iv: "2b334853cefcfc5b7a7b6f08cf40b8cd"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.950310 843 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:34.951052 857 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:34.951282 843 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:34.951382 843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "1c32d3c107854ef8a59239015704099a"
format_stamp: "Formatted at 2026-09-27 18:47:34 on dist-test-slave-vs05"
server_key: "f751287d681e11a9093287118cbaf271"
server_key_iv: "2b334853cefcfc5b7a7b6f08cf40b8cd"
server_key_version: "encryptionkey@0"
I20260927 18:47:34.951473 843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:34.970034 843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:34.970374 843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:34.970536 843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:34.974634 843 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:44641
I20260927 18:47:34.974686 909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:44641 every 8 connection(s)
I20260927 18:47:34.975065 843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:34.976034 910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:34.978471 910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: Bootstrap starting.
I20260927 18:47:34.978931 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 843
I20260927 18:47:34.979048 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:34.979147 910 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:34.979290 15803 external_mini_cluster.cc:1428] Setting key dd7b025742343b832318ad3ba690d85b
I20260927 18:47:34.979523 910 log.cc:824] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:34.980212 910 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: No bootstrap required, opened a new log
I20260927 18:47:34.982085 910 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } }
I20260927 18:47:34.982219 910 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:34.982242 910 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c32d3c107854ef8a59239015704099a, State: Initialized, Role: FOLLOWER
I20260927 18:47:34.982385 910 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } }
I20260927 18:47:34.982458 910 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:34.982493 910 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:34.982547 910 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:34.983103 910 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } }
I20260927 18:47:34.983248 910 leader_election.cc:302] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1c32d3c107854ef8a59239015704099a; no voters:
I20260927 18:47:34.983464 910 leader_election.cc:288] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:34.983489 915 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:34.983667 915 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [term 1 LEADER]: Becoming Leader. State: Replica: 1c32d3c107854ef8a59239015704099a, State: Running, Role: LEADER
I20260927 18:47:34.983757 915 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } }
I20260927 18:47:34.983783 910 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:34.984068 915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1c32d3c107854ef8a59239015704099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } } }
I20260927 18:47:34.984133 915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:34.984262 915 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1c32d3c107854ef8a59239015704099a. Latest consensus state: current_term: 1 leader_uuid: "1c32d3c107854ef8a59239015704099a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1c32d3c107854ef8a59239015704099a" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44641 } } }
I20260927 18:47:34.984344 915 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:34.984428 920 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:34.985185 920 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:34.985635 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:34.979820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60938 (local address 127.15.110.254:44641)
0927 18:47:34.980027 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:34.980035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:34.980070 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:47:34.980532 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:34.980540 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:34.980606 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:34.980750 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:34.980990 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.981855 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.982893 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:34.983422 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:34.983551 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:34.984520 (+ 969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:34.984525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:34.984525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:34.984728 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:34.984771 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:34.984786 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:34.984870 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:34.985176 (+ 306us) server_negotiation.cc:299] Negotiation successful
0927 18:47:34.985416 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260927 18:47:34.986836 920 catalog_manager.cc:1347] Generated new cluster ID: 469f8dd01f3e423b895cc787732d9c67
I20260927 18:47:34.986891 920 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:34.995692 920 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:34.996320 920 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:35.000625 920 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: Generated new TSK 0
I20260927 18:47:35.000846 920 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:35.002768 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--builtin_ntp_servers=127.15.110.212:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:35.090713 934 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:35.090948 934 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:35.090987 934 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:35.092253 934 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:35.092329 934 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:35.092374 934 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:35.093760 934 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.934
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:35.094022 934 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:35.094331 934 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:35.100327 940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:35.100538 934 server_base.cc:1034] running on GCE node
W20260927 18:47:35.100234 939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:35.100660 942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:35.100886 934 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:35.101186 934 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:35.102372 934 hybrid_clock.cc:648] HybridClock initialized: now 1790534855102362 us; error 31 us; skew 500 ppm
I20260927 18:47:35.103917 934 webserver.cc:460] Webserver started at http://127.15.110.193:34297/ using document root <none> and password file <none>
I20260927 18:47:35.104194 934 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:35.104256 934 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:35.104396 934 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:35.105778 934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "fc23185b924147da98d963857f2717b4"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "fe87c8e86e07ceaace328b824fcde58c"
server_key_iv: "73e2d99561d8b4a1dd7bfa32c2018d1e"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.106190 934 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "fc23185b924147da98d963857f2717b4"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "fe87c8e86e07ceaace328b824fcde58c"
server_key_iv: "73e2d99561d8b4a1dd7bfa32c2018d1e"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.107862 934 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260927 18:47:35.108686 948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.108896 934 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:35.108999 934 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "fc23185b924147da98d963857f2717b4"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "fe87c8e86e07ceaace328b824fcde58c"
server_key_iv: "73e2d99561d8b4a1dd7bfa32c2018d1e"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.109077 934 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:35.126776 934 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:35.127080 934 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:35.127266 934 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:35.127506 934 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:35.127818 934 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:35.127851 934 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.127873 934 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:35.127921 934 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.133522 934 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:35253
I20260927 18:47:35.133620 1061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:35253 every 8 connection(s)
I20260927 18:47:35.133924 934 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:35.138674 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.134647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:34245 (local address 127.15.110.254:44641)
0927 18:47:35.134723 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.134726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.134907 (+ 181us) server_negotiation.cc:407] Connection header received
0927 18:47:35.135381 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.135382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.135392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.135442 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:35.136067 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.136543 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.137471 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.137646 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.137714 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.138435 (+ 721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:35.138437 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:35.138437 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:35.138442 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:35.138455 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:35.138458 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:35.138501 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.138596 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.138616 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260927 18:47:35.138872 1064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.134611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44641 (local address 127.15.110.193:34245)
0927 18:47:35.134791 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.134824 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.135298 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.135501 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.135504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.135513 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:35.135997 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.136002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.136624 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.136625 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.137398 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.137400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.137723 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.137724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.137824 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.138303 (+ 479us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:35.138303 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:35.138330 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:35.138352 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:35.138514 (+ 162us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:35.138518 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:35.138521 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.138572 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.138646 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":101,"threads_started":1}
I20260927 18:47:35.139473 1062 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44641
I20260927 18:47:35.139565 1062 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:35.139747 1062 heartbeater.cc:511] Master 127.15.110.254:44641 requested a full tablet report, sending...
I20260927 18:47:35.140276 874 ts_manager.cc:194] Registered new tserver with Master: fc23185b924147da98d963857f2717b4 (127.15.110.193:35253)
I20260927 18:47:35.140875 874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.193:34245
I20260927 18:47:35.141140 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 934
I20260927 18:47:35.141258 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:35.141460 15803 external_mini_cluster.cc:1428] Setting key d4ade2c2442de480e418a1a865e7cfa6
I20260927 18:47:35.142283 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--builtin_ntp_servers=127.15.110.212:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:35.221697 1065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:35.221890 1065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:35.221910 1065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:35.222972 1065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:35.223003 1065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:35.223028 1065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:35.224490 1065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:35.224715 1065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:35.225016 1065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:35.230075 1073 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:35.230108 1071 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:35.230108 1070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:35.230371 1065 server_base.cc:1034] running on GCE node
I20260927 18:47:35.230579 1065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:35.230805 1065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:35.231967 1065 hybrid_clock.cc:648] HybridClock initialized: now 1790534855231964 us; error 36 us; skew 500 ppm
I20260927 18:47:35.233155 1065 webserver.cc:460] Webserver started at http://127.15.110.194:37941/ using document root <none> and password file <none>
I20260927 18:47:35.233431 1065 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:35.233498 1065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:35.233634 1065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:35.234654 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "c94ca426a79e8fd1f57131d20ad230c5"
server_key_iv: "f9ae268a0d355a8d834161c1da8c1d41"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.235014 1065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "c94ca426a79e8fd1f57131d20ad230c5"
server_key_iv: "f9ae268a0d355a8d834161c1da8c1d41"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.236366 1065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:35.237069 1079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.237241 1065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:35.237342 1065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "c94ca426a79e8fd1f57131d20ad230c5"
server_key_iv: "f9ae268a0d355a8d834161c1da8c1d41"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.237430 1065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:35.254988 1065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:35.255398 1065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:35.255549 1065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:35.255771 1065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:35.256084 1065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:35.256137 1065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.256179 1065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:35.256207 1065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.261891 1065 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:44623
I20260927 18:47:35.261933 1192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:44623 every 8 connection(s)
I20260927 18:47:35.262244 1065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:35.266835 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1065
I20260927 18:47:35.266921 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:35.267138 15803 external_mini_cluster.cc:1428] Setting key e3668e0c8db4a5fbdf5b1bf820f81aef
I20260927 18:47:35.267596 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.262987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:51153 (local address 127.15.110.254:44641)
0927 18:47:35.263047 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.263049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.263341 (+ 292us) server_negotiation.cc:407] Connection header received
0927 18:47:35.263893 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.263895 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.263907 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.263959 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:35.264645 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.265153 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.266140 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.266349 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.266425 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.267319 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:35.267320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:35.267321 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:35.267326 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:35.267343 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:35.267348 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:35.267408 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.267493 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.267520 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:47:35.267832 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.262962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44641 (local address 127.15.110.194:51153)
0927 18:47:35.263162 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.263237 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.263779 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.264046 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.264050 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.264066 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:35.264554 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.264559 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.265258 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.265259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.266040 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.266042 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.266432 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.266433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.266572 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.267130 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:35.267131 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:35.267159 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:35.267222 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:35.267421 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:35.267426 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:35.267429 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.267501 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.267617 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260927 18:47:35.268186 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--builtin_ntp_servers=127.15.110.212:42497
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:47:35.268254 1193 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44641
I20260927 18:47:35.268340 1193 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:35.268512 1193 heartbeater.cc:511] Master 127.15.110.254:44641 requested a full tablet report, sending...
I20260927 18:47:35.268873 874 ts_manager.cc:194] Registered new tserver with Master: 5a4e9a49816b4b5d97cd82f8a184bb8a (127.15.110.194:44623)
I20260927 18:47:35.269246 874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.194:51153
W20260927 18:47:35.348351 1196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:35.348548 1196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:35.348569 1196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:35.349706 1196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:35.349746 1196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:35.349794 1196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:35.351102 1196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:42497
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:35.351370 1196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:35.351612 1196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:35.356689 1204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:35.356736 1201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:35.356683 1202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:35.356986 1196 server_base.cc:1034] running on GCE node
I20260927 18:47:35.357172 1196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:35.357424 1196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:35.358594 1196 hybrid_clock.cc:648] HybridClock initialized: now 1790534855358576 us; error 35 us; skew 500 ppm
I20260927 18:47:35.359805 1196 webserver.cc:460] Webserver started at http://127.15.110.195:42269/ using document root <none> and password file <none>
I20260927 18:47:35.360047 1196 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:35.360092 1196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:35.360244 1196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:35.361334 1196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "340a2003c817493fad131be0d790e5ac"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "74a10d3b017bf9c202e311b60d1ede3f"
server_key_iv: "d336d8ef56df7da85299c29c6f8cfab3"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.361689 1196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "340a2003c817493fad131be0d790e5ac"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "74a10d3b017bf9c202e311b60d1ede3f"
server_key_iv: "d336d8ef56df7da85299c29c6f8cfab3"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.362962 1196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:35.363734 1210 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.363935 1196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.364032 1196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "340a2003c817493fad131be0d790e5ac"
format_stamp: "Formatted at 2026-09-27 18:47:35 on dist-test-slave-vs05"
server_key: "74a10d3b017bf9c202e311b60d1ede3f"
server_key_iv: "d336d8ef56df7da85299c29c6f8cfab3"
server_key_version: "encryptionkey@0"
I20260927 18:47:35.364118 1196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:35.382113 1196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:35.382471 1196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:35.382650 1196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:35.382896 1196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:35.383329 1196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:35.383383 1196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.383422 1196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:35.383448 1196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:35.389248 1196 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40549
I20260927 18:47:35.389324 1323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40549 every 8 connection(s)
I20260927 18:47:35.389662 1196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:47:35.393074 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1196
I20260927 18:47:35.393162 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:47:35.393349 15803 external_mini_cluster.cc:1428] Setting key 5e8b27112b51d3e828c93b9c2734f415
I20260927 18:47:35.394898 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.390377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:43027 (local address 127.15.110.254:44641)
0927 18:47:35.390422 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.390424 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.390683 (+ 259us) server_negotiation.cc:407] Connection header received
0927 18:47:35.391159 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.391160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.391206 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.391278 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:35.391907 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.392383 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.393406 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.393643 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.393724 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.394640 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:35.394641 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:35.394641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:35.394646 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:35.394660 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:35.394664 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:35.394704 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.394782 (+ 78us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.394807 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:47:35.395115 1326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.390389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44641 (local address 127.15.110.195:43027)
0927 18:47:35.390559 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.390592 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.391073 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.391339 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.391343 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.391352 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:35.391836 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.391840 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.392461 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.392462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.393322 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.393325 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.393743 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.393745 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.393906 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.394492 (+ 586us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:35.394493 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:35.394521 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:35.394543 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:35.394715 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:35.394719 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:35.394722 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.394785 (+ 63us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.394885 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":87,"threads_started":1}
I20260927 18:47:35.395572 1324 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44641
I20260927 18:47:35.395675 1324 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:35.395867 1324 heartbeater.cc:511] Master 127.15.110.254:44641 requested a full tablet report, sending...
I20260927 18:47:35.396219 874 ts_manager.cc:194] Registered new tserver with Master: 340a2003c817493fad131be0d790e5ac (127.15.110.195:40549)
I20260927 18:47:35.396600 874 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.195:43027
I20260927 18:47:35.405191 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:35.409350 914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.406522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60944 (local address 127.15.110.254:44641)
0927 18:47:35.406580 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.406582 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.406688 (+ 106us) server_negotiation.cc:407] Connection header received
0927 18:47:35.406744 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.406745 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.406753 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.406792 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:35.406938 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.407612 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.408328 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.408516 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.408568 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.409144 (+ 576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:35.409145 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:35.409145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:35.409150 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:35.409162 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:35.409166 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:35.409209 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.409275 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.409295 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:47:35.410916 874 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60944:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:35.412595 874 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:35.418351 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37430 (local address 127.15.110.195:40549)
0927 18:47:35.415050 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.415055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.415099 (+ 44us) server_negotiation.cc:407] Connection header received
0927 18:47:35.415179 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.415182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.415226 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.415332 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:35.415494 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416314 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.417317 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.417964 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418053 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.418140 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.418222 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.418259 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":96,"threads_started":1}
I20260927 18:47:35.418349 1332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40549 (local address 127.0.0.1:37430)
0927 18:47:35.415024 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.415035 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.415105 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.415341 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.415344 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.415345 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:35.415411 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.415414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416403 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.416404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.417217 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.417219 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.418053 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418054 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.418131 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.418144 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.418213 (+ 69us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.418265 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":4,"thread_start_us":86,"threads_started":1}
I20260927 18:47:35.419162 1333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44623 (local address 127.0.0.1:35022)
0927 18:47:35.414941 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.414987 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.415051 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.415381 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.415382 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.415384 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:35.415432 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.415434 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416843 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.416844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.418043 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.418045 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.418934 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.418995 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.418999 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.419057 (+ 58us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.419096 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":52,"thread_start_us":54,"threads_started":1}
I20260927 18:47:35.419216 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35022 (local address 127.15.110.194:44623)
0927 18:47:35.415125 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.415129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.415139 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:35.415203 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.415205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.415222 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.415300 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:35.415609 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416747 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418135 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.418843 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418936 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.419009 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.419065 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.419100 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":81,"threads_started":1}
I20260927 18:47:35.419327 1258 tablet_service.cc:1467] Processing CreateTablet for tablet 43260148f2384f2a8d782d9a376faafe (DEFAULT_TABLE table=test-table [id=6fc7f9e378a94095a8ca925c07f70961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:35.419675 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:35253 (local address 127.0.0.1:60716)
0927 18:47:35.415159 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.415168 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.415228 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.415427 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.415428 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.415430 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:35.415467 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.415468 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416843 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.416843 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.418112 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.418114 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.419441 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.419442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.419488 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.419490 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.419566 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.419608 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":38,"thread_start_us":96,"threads_started":1}
I20260927 18:47:35.419697 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.414909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60716 (local address 127.15.110.193:35253)
0927 18:47:35.415286 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.415290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.415299 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:47:35.415339 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.415341 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.415356 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.415432 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:35.415529 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.416745 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.418191 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.419243 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.419316 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.419399 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.419583 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.419619 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":100,"threads_started":1}
I20260927 18:47:35.419737 1258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43260148f2384f2a8d782d9a376faafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:35.419920 1127 tablet_service.cc:1467] Processing CreateTablet for tablet 43260148f2384f2a8d782d9a376faafe (DEFAULT_TABLE table=test-table [id=6fc7f9e378a94095a8ca925c07f70961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:35.420230 1127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43260148f2384f2a8d782d9a376faafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:35.423167 1338 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Bootstrap starting.
I20260927 18:47:35.423929 1338 tablet_bootstrap.cc:654] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:35.424008 1339 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Bootstrap starting.
I20260927 18:47:35.424034 996 tablet_service.cc:1467] Processing CreateTablet for tablet 43260148f2384f2a8d782d9a376faafe (DEFAULT_TABLE table=test-table [id=6fc7f9e378a94095a8ca925c07f70961]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:35.424237 1338 log.cc:824] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:35.424403 996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43260148f2384f2a8d782d9a376faafe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:35.424655 1339 tablet_bootstrap.cc:654] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:35.424990 1339 log.cc:824] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:35.425135 1338 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: No bootstrap required, opened a new log
I20260927 18:47:35.425207 1338 ts_tablet_manager.cc:1397] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:47:35.427714 1339 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: No bootstrap required, opened a new log
I20260927 18:47:35.427788 1339 ts_tablet_manager.cc:1397] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260927 18:47:35.427940 1342 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: Bootstrap starting.
I20260927 18:47:35.427874 1338 raft_consensus.cc:348] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.428040 1338 raft_consensus.cc:374] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:35.428085 1338 raft_consensus.cc:729] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 340a2003c817493fad131be0d790e5ac, State: Initialized, Role: FOLLOWER
I20260927 18:47:35.428193 1338 consensus_queue.cc:260] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.428438 1338 ts_tablet_manager.cc:1428] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:35.428467 1324 heartbeater.cc:503] Master 127.15.110.254:44641 was elected leader, sending a full tablet report...
I20260927 18:47:35.428608 1342 tablet_bootstrap.cc:654] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:35.428969 1342 log.cc:824] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:35.429715 1342 tablet_bootstrap.cc:492] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: No bootstrap required, opened a new log
I20260927 18:47:35.429797 1342 ts_tablet_manager.cc:1397] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:35.430351 1339 raft_consensus.cc:348] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.430473 1339 raft_consensus.cc:374] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:35.430495 1339 raft_consensus.cc:729] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a4e9a49816b4b5d97cd82f8a184bb8a, State: Initialized, Role: FOLLOWER
I20260927 18:47:35.430612 1339 consensus_queue.cc:260] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.430856 1339 ts_tablet_manager.cc:1428] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:35.430943 1193 heartbeater.cc:503] Master 127.15.110.254:44641 was elected leader, sending a full tablet report...
I20260927 18:47:35.432052 1342 raft_consensus.cc:348] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.432166 1342 raft_consensus.cc:374] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:35.432186 1342 raft_consensus.cc:729] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc23185b924147da98d963857f2717b4, State: Initialized, Role: FOLLOWER
I20260927 18:47:35.432312 1342 consensus_queue.cc:260] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.432552 1342 ts_tablet_manager.cc:1428] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:35.432605 1062 heartbeater.cc:503] Master 127.15.110.254:44641 was elected leader, sending a full tablet report...
W20260927 18:47:35.512755 1194 tablet.cc:2369] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:35.540318 1345 raft_consensus.cc:482] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:35.540443 1345 raft_consensus.cc:504] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.540879 1345 leader_election.cc:288] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 340a2003c817493fad131be0d790e5ac (127.15.110.195:40549), fc23185b924147da98d963857f2717b4 (127.15.110.193:35253)
I20260927 18:47:35.545030 1195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.541101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:35253 (local address 127.15.110.194:50711)
0927 18:47:35.541149 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.541162 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.541207 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.541440 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.541441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.541443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:35.541516 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.541519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.542481 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.542483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.543871 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.543874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.544742 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.544823 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.544829 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.544890 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.544937 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":99}
I20260927 18:47:35.545049 1337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.541161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:50711 (local address 127.15.110.193:35253)
0927 18:47:35.541227 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.541230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.541270 (+ 40us) server_negotiation.cc:407] Connection header received
0927 18:47:35.541301 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.541301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.541313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.541406 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:35.541608 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.542380 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544000 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.544640 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544735 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.544743 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.544924 (+ 181us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.544968 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:47:35.545046 1335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.541166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:40527 (local address 127.15.110.195:40549)
0927 18:47:35.541227 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.541230 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.541390 (+ 160us) server_negotiation.cc:407] Connection header received
0927 18:47:35.541422 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.541423 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.541432 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.541486 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:35.541611 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.542374 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544000 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.544640 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544734 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.544743 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.544923 (+ 180us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.544968 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:47:35.545037 1347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.541102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40549 (local address 127.15.110.194:40527)
0927 18:47:35.541298 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.541307 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:35.541357 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:35.541486 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:35.541487 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:35.541488 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:35.541547 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.541548 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.542481 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.542483 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.543871 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:35.543874 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.544743 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.544744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:35.544823 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.544829 (+ 6us) client_negotiation.cc:769] Sending connection context
0927 18:47:35.544902 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:47:35.544937 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":103,"thread_start_us":100,"threads_started":1}
I20260927 18:47:35.545454 1016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43260148f2384f2a8d782d9a376faafe" candidate_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc23185b924147da98d963857f2717b4" is_pre_election: true
I20260927 18:47:35.545620 1016 raft_consensus.cc:2454] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a4e9a49816b4b5d97cd82f8a184bb8a in term 0.
I20260927 18:47:35.545557 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43260148f2384f2a8d782d9a376faafe" candidate_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340a2003c817493fad131be0d790e5ac" is_pre_election: true
I20260927 18:47:35.545807 1278 raft_consensus.cc:2454] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a4e9a49816b4b5d97cd82f8a184bb8a in term 0.
I20260927 18:47:35.545884 1083 leader_election.cc:302] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a4e9a49816b4b5d97cd82f8a184bb8a, fc23185b924147da98d963857f2717b4; no voters:
I20260927 18:47:35.546106 1345 raft_consensus.cc:2781] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:35.546176 1345 raft_consensus.cc:482] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:35.546195 1345 raft_consensus.cc:3037] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:35.546998 1345 raft_consensus.cc:504] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.547168 1345 leader_election.cc:288] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [CANDIDATE]: Term 1 election: Requested vote from peers 340a2003c817493fad131be0d790e5ac (127.15.110.195:40549), fc23185b924147da98d963857f2717b4 (127.15.110.193:35253)
I20260927 18:47:35.547339 1278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43260148f2384f2a8d782d9a376faafe" candidate_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "340a2003c817493fad131be0d790e5ac"
I20260927 18:47:35.547405 1016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43260148f2384f2a8d782d9a376faafe" candidate_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc23185b924147da98d963857f2717b4"
I20260927 18:47:35.547432 1278 raft_consensus.cc:3037] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:35.547475 1016 raft_consensus.cc:3037] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:35.548372 1016 raft_consensus.cc:2454] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a4e9a49816b4b5d97cd82f8a184bb8a in term 1.
I20260927 18:47:35.548365 1278 raft_consensus.cc:2454] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a4e9a49816b4b5d97cd82f8a184bb8a in term 1.
I20260927 18:47:35.548538 1080 leader_election.cc:302] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 340a2003c817493fad131be0d790e5ac, 5a4e9a49816b4b5d97cd82f8a184bb8a; no voters:
I20260927 18:47:35.548631 1345 raft_consensus.cc:2781] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:35.548823 1345 raft_consensus.cc:686] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 1 LEADER]: Becoming Leader. State: Replica: 5a4e9a49816b4b5d97cd82f8a184bb8a, State: Running, Role: LEADER
I20260927 18:47:35.548946 1345 consensus_queue.cc:237] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } }
I20260927 18:47:35.549871 874 catalog_manager.cc:5495] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a4e9a49816b4b5d97cd82f8a184bb8a (127.15.110.194). New cstate: current_term: 1 leader_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5a4e9a49816b4b5d97cd82f8a184bb8a" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44623 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 } health_report { overall_health: UNKNOWN } } }
I20260927 18:47:35.566393 1336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.564273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.15.110.194:44623)
0927 18:47:35.564338 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:35.564340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:35.564363 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:47:35.564399 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:35.564400 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:35.564417 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:35.564470 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:35.564585 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.565057 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.565734 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:35.565912 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:35.565959 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:35.566093 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:35.566204 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:35.566266 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:35.566301 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 18:47:35.566316 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:47:35.568369 1016 raft_consensus.cc:1261] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 1 FOLLOWER]: Refusing update from remote peer 5a4e9a49816b4b5d97cd82f8a184bb8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:35.568361 1278 raft_consensus.cc:1261] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 1 FOLLOWER]: Refusing update from remote peer 5a4e9a49816b4b5d97cd82f8a184bb8a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:35.568722 1345 consensus_queue.cc:1035] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc23185b924147da98d963857f2717b4" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 35253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:35.568863 1348 consensus_queue.cc:1035] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [LEADER]: Connected to new peer: Peer: permanent_uuid: "340a2003c817493fad131be0d790e5ac" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:35.571197 1351 mvcc.cc:205] Tried to move back new op lower bound from 7334030768405078016 to 7334030768329347072. Current Snapshot: MvccSnapshot[applied={T|T < 7334030768405078016}]
I20260927 18:47:35.571848 1352 mvcc.cc:205] Tried to move back new op lower bound from 7334030768405078016 to 7334030768329347072. Current Snapshot: MvccSnapshot[applied={T|T < 7334030768405078016}]
I20260927 18:47:35.572016 1353 mvcc.cc:205] Tried to move back new op lower bound from 7334030768405078016 to 7334030768329347072. Current Snapshot: MvccSnapshot[applied={T|T < 7334030768405078016}]
I20260927 18:47:35.573653 874 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:35.573731 874 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60944:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:35.574470 874 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1c32d3c107854ef8a59239015704099a: Sending DeleteTablet for 3 replicas of tablet 43260148f2384f2a8d782d9a376faafe
I20260927 18:47:35.574818 996 tablet_service.cc:1514] Processing DeleteTablet for tablet 43260148f2384f2a8d782d9a376faafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:35 UTC) from {username='slave'} at 127.0.0.1:60716
I20260927 18:47:35.574859 1127 tablet_service.cc:1514] Processing DeleteTablet for tablet 43260148f2384f2a8d782d9a376faafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:35 UTC) from {username='slave'} at 127.0.0.1:35022
I20260927 18:47:35.574875 1258 tablet_service.cc:1514] Processing DeleteTablet for tablet 43260148f2384f2a8d782d9a376faafe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:35 UTC) from {username='slave'} at 127.0.0.1:37430
I20260927 18:47:35.575075 1360 tablet_replica.cc:330] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: stopping tablet replica
I20260927 18:47:35.575168 1360 raft_consensus.cc:2229] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:35.575276 1362 tablet_replica.cc:330] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4: stopping tablet replica
I20260927 18:47:35.575284 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 934
I20260927 18:47:35.575321 1360 raft_consensus.cc:2258] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:35.575354 1362 raft_consensus.cc:2229] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:35.575435 1362 raft_consensus.cc:2258] T 43260148f2384f2a8d782d9a376faafe P fc23185b924147da98d963857f2717b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:35.575563 1361 tablet_replica.cc:330] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: stopping tablet replica
I20260927 18:47:35.575634 1361 raft_consensus.cc:2229] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:47:35.575726 1361 raft_consensus.cc:2258] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:35.576159 1361 ts_tablet_manager.cc:1905] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:35.577716 1361 ts_tablet_manager.cc:1918] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:35.577765 1361 log.cc:1196] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/43260148f2384f2a8d782d9a376faafe
I20260927 18:47:35.577946 1361 ts_tablet_manager.cc:1939] T 43260148f2384f2a8d782d9a376faafe P 5a4e9a49816b4b5d97cd82f8a184bb8a: Deleting consensus metadata
I20260927 18:47:35.578251 860 catalog_manager.cc:4841] TS 5a4e9a49816b4b5d97cd82f8a184bb8a (127.15.110.194:44623): tablet 43260148f2384f2a8d782d9a376faafe (table test-table [id=6fc7f9e378a94095a8ca925c07f70961]) successfully deleted
I20260927 18:47:35.579627 1360 ts_tablet_manager.cc:1905] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:35.581117 1360 ts_tablet_manager.cc:1918] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:35.581204 1360 log.cc:1196] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/43260148f2384f2a8d782d9a376faafe
I20260927 18:47:35.581434 1360 ts_tablet_manager.cc:1939] T 43260148f2384f2a8d782d9a376faafe P 340a2003c817493fad131be0d790e5ac: Deleting consensus metadata
I20260927 18:47:35.581820 858 catalog_manager.cc:4841] TS 340a2003c817493fad131be0d790e5ac (127.15.110.195:40549): tablet 43260148f2384f2a8d782d9a376faafe (table test-table [id=6fc7f9e378a94095a8ca925c07f70961]) successfully deleted
W20260927 18:47:35.582319 861 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:35253 (error 108)
I20260927 18:47:35.582777 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1065
I20260927 18:47:35.582788 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:35.582655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:35253 (local address 127.0.0.1:60728)
0927 18:47:35.582713 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:35.582761 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:35253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:47:35.583581 861 catalog_manager.cc:4584] TS fc23185b924147da98d963857f2717b4 (127.15.110.193:35253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43260148f2384f2a8d782d9a376faafe: Network error: Client connection negotiation failed: client connection to 127.15.110.193:35253: connect: Connection refused (error 111)
I20260927 18:47:35.588405 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1196
I20260927 18:47:35.593411 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 843
2026-09-27T18:47:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (766 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:35 dist-test-slave-vs05 krb5kdc[1366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:35 dist-test-slave-vs05 krb5kdc[1366](info): set up 2 sockets
Sep 27 18:47:35 dist-test-slave-vs05 krb5kdc[1366](info): commencing operation
krb5kdc: starting...
W20260927 18:47:36.630878 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.004s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:36 dist-test-slave-vs05 krb5kdc[1366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:47:36.750655 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36229
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36229
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:36.832201 1379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:36.832451 1379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:36.832497 1379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:36.833631 1379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:36.833711 1379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:36.833739 1379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:36.833770 1379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:36.833796 1379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:36.835304 1379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36229
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36229
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:36.835583 1379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:36.835820 1379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:36.840644 1385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:36.840613 1387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:36.840622 1384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:36.840888 1379 server_base.cc:1034] running on GCE node
I20260927 18:47:36.841126 1379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:36.841434 1379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:36.842605 1379 hybrid_clock.cc:648] HybridClock initialized: now 1790534856842584 us; error 51 us; skew 500 ppm
Sep 27 18:47:36 dist-test-slave-vs05 krb5kdc[1366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534856, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:36.850684 1379 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:36.851131 1379 webserver.cc:460] Webserver started at http://127.15.110.254:44289/ using document root <none> and password file <none>
I20260927 18:47:36.851401 1379 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:36.851473 1379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:36.851625 1379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:36.852710 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "2163bc8265f74416a82a83a13e576771"
format_stamp: "Formatted at 2026-09-27 18:47:36 on dist-test-slave-vs05"
server_key: "508cd6ccaae7ccad29590294eba862f6"
server_key_iv: "5d1126f0ebc5c9a469cc927a70d5f65b"
server_key_version: "encryptionkey@0"
I20260927 18:47:36.853072 1379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "2163bc8265f74416a82a83a13e576771"
format_stamp: "Formatted at 2026-09-27 18:47:36 on dist-test-slave-vs05"
server_key: "508cd6ccaae7ccad29590294eba862f6"
server_key_iv: "5d1126f0ebc5c9a469cc927a70d5f65b"
server_key_version: "encryptionkey@0"
I20260927 18:47:36.854445 1379 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:36.855147 1394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:36.855362 1379 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:36.855459 1379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "2163bc8265f74416a82a83a13e576771"
format_stamp: "Formatted at 2026-09-27 18:47:36 on dist-test-slave-vs05"
server_key: "508cd6ccaae7ccad29590294eba862f6"
server_key_iv: "5d1126f0ebc5c9a469cc927a70d5f65b"
server_key_version: "encryptionkey@0"
I20260927 18:47:36.855516 1379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:36.866887 1379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:36.869110 1379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:36.869220 1379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:36.873054 1379 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:36229
I20260927 18:47:36.873085 1446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:36229 every 8 connection(s)
I20260927 18:47:36.873442 1379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:36.874331 1447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:36.875355 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1379
I20260927 18:47:36.875474 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:36.875682 15803 external_mini_cluster.cc:1428] Setting key 7aa6fce680cde687037328bec18248dc
I20260927 18:47:36.877847 1447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: Bootstrap starting.
I20260927 18:47:36.878429 1447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:36.878722 1447 log.cc:824] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:36.879513 1447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: No bootstrap required, opened a new log
I20260927 18:47:36.882051 1447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } }
I20260927 18:47:36.882226 1447 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:36.882262 1447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2163bc8265f74416a82a83a13e576771, State: Initialized, Role: FOLLOWER
I20260927 18:47:36.882423 1447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } }
I20260927 18:47:36.882504 1447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:36.882536 1447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:36.882583 1447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 0 FOLLOWER]: Advancing to term 1
Sep 27 18:47:36 dist-test-slave-vs05 krb5kdc[1366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534856, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:36.883332 1447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } }
I20260927 18:47:36.883421 1447 leader_election.cc:302] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2163bc8265f74416a82a83a13e576771; no voters:
I20260927 18:47:36.883598 1447 leader_election.cc:288] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:36.883776 1453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:36.883883 1447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:36.883934 1453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [term 1 LEADER]: Becoming Leader. State: Replica: 2163bc8265f74416a82a83a13e576771, State: Running, Role: LEADER
I20260927 18:47:36.884042 1453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } }
I20260927 18:47:36.884357 1453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2163bc8265f74416a82a83a13e576771. Latest consensus state: current_term: 1 leader_uuid: "2163bc8265f74416a82a83a13e576771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } } }
I20260927 18:47:36.884440 1453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:36.884701 1452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2163bc8265f74416a82a83a13e576771" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2163bc8265f74416a82a83a13e576771" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36229 } } }
I20260927 18:47:36.884857 1452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771 [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:36.885689 1467 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:36.885743 1467 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:36.885797 1460 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:36.885983 1460 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:36.887421 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:36.876229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52898 (local address 127.15.110.254:36229)
0927 18:47:36.876399 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:36.876408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:36.876433 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:47:36.876849 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:36.876856 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:36.876910 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:36.877021 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:36.877795 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:36.878859 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:36.879857 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:36.880259 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:36.880348 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:36.883203 (+ 2855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:36.883209 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:36.883209 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:36.883222 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:36.885555 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:36.886014 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:36.886017 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:36.886018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:36.886129 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:36.886399 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:36.886399 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:36.886400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:36.886728 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:36.886875 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:36.887015 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:47:36.887171 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":87,"threads_started":1}
I20260927 18:47:36.887668 1460 catalog_manager.cc:1347] Generated new cluster ID: 9f42484e21774d4d97cc18402eec0dd7
I20260927 18:47:36.887710 1460 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:36.899582 1460 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:36.900086 1460 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:36.906126 1460 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: Generated new TSK 0
I20260927 18:47:36.906327 1460 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:47:36.938095 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:37.026976 1473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:37.027231 1473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:37.027290 1473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:37.028432 1473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:37.028470 1473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:37.028496 1473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:37.029934 1473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:37.030164 1473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:37.030408 1473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:37.036180 1478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.036288 1481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.036239 1479 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.036441 1473 server_base.cc:1034] running on GCE node
I20260927 18:47:37.036685 1473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:37.036983 1473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:37.038143 1473 hybrid_clock.cc:648] HybridClock initialized: now 1790534857038129 us; error 34 us; skew 500 ppm
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:37.040702 1473 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:47:37.041141 1473 webserver.cc:460] Webserver started at http://127.15.110.193:45505/ using document root <none> and password file <none>
I20260927 18:47:37.041364 1473 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:37.041432 1473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:37.041579 1473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:37.042603 1473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "336e0b643c8d43c7a65158f9c0ca3001"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "c27d5262465d297b1d925600b0b247c0"
server_key_iv: "7ad4d57c480737b02812f04f0006db76"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.042999 1473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "336e0b643c8d43c7a65158f9c0ca3001"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "c27d5262465d297b1d925600b0b247c0"
server_key_iv: "7ad4d57c480737b02812f04f0006db76"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.044358 1473 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:37.045073 1488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.045252 1473 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:37.045348 1473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "336e0b643c8d43c7a65158f9c0ca3001"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "c27d5262465d297b1d925600b0b247c0"
server_key_iv: "7ad4d57c480737b02812f04f0006db76"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.045428 1473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:37.063488 1473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:37.065762 1473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:37.065932 1473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:37.066164 1473 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:37.066459 1473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:37.066514 1473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.066555 1473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:37.066583 1473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.072232 1473 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:42953
I20260927 18:47:37.072281 1601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:42953 every 8 connection(s)
I20260927 18:47:37.072574 1473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:37.073530 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1473
I20260927 18:47:37.073653 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:37.073863 15803 external_mini_cluster.cc:1428] Setting key e85778486c77035137b87c2a9a986dea
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
I20260927 18:47:37.083765 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.073238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:42221 (local address 127.15.110.254:36229)
0927 18:47:37.073290 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.073292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.073539 (+ 247us) server_negotiation.cc:407] Connection header received
0927 18:47:37.074325 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.074326 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.074337 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.074405 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:37.075750 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.076315 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.077854 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.078058 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.078123 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.080643 (+ 2520us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:37.080645 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:37.080645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:37.080652 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:37.082121 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.082710 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.082712 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.082712 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.082773 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.083054 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.083055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.083055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.083170 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:37.083272 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.083543 (+ 271us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.083663 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:47:37.083995 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.073177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36229 (local address 127.15.110.193:42221)
0927 18:47:37.073369 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.073421 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.074204 (+ 783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.074467 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.074471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.075356 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:37.075632 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.075638 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.076394 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.076395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.077744 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.077747 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.078160 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.078161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.078327 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.078850 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:37.078861 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:37.080535 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:37.082257 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.082312 (+ 55us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.082314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.082606 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.082848 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.082849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.082850 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.082984 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.083262 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:37.083269 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:37.083411 (+ 142us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.083522 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.083730 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":68,"threads_started":1}
I20260927 18:47:37.084596 1602 heartbeater.cc:347] Connected to a master server at 127.15.110.254:36229
I20260927 18:47:37.084697 1602 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:37.084889 1602 heartbeater.cc:511] Master 127.15.110.254:36229 requested a full tablet report, sending...
I20260927 18:47:37.085516 1411 ts_manager.cc:194] Registered new tserver with Master: 336e0b643c8d43c7a65158f9c0ca3001 (127.15.110.193:42953)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
I20260927 18:47:37.086210 1411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:42221
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:47:37.102284 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:37.183595 1607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:37.183813 1607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:37.183840 1607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:37.185246 1607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:37.185289 1607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:37.185318 1607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:37.186858 1607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:37.187071 1607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:37.187333 1607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:37.192355 1615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.192355 1612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.192369 1613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.192597 1607 server_base.cc:1034] running on GCE node
I20260927 18:47:37.192907 1607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:37.193109 1607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:37.194269 1607 hybrid_clock.cc:648] HybridClock initialized: now 1790534857194267 us; error 35 us; skew 500 ppm
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:37.196787 1607 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:47:37.197218 1607 webserver.cc:460] Webserver started at http://127.15.110.194:44985/ using document root <none> and password file <none>
I20260927 18:47:37.197456 1607 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:37.197525 1607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:37.197671 1607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:37.198694 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "47627083594e40b98acd6e25f8b37d66"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "af5c6aba0352c0c2f0d9545de843b0b3"
server_key_iv: "03d39257d366b9f706aca88282b552b1"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.199143 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "47627083594e40b98acd6e25f8b37d66"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "af5c6aba0352c0c2f0d9545de843b0b3"
server_key_iv: "03d39257d366b9f706aca88282b552b1"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.200961 1607 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:37.201858 1622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.202059 1607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:37.202159 1607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "47627083594e40b98acd6e25f8b37d66"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "af5c6aba0352c0c2f0d9545de843b0b3"
server_key_iv: "03d39257d366b9f706aca88282b552b1"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.202250 1607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:37.216063 1607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:37.218191 1607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:37.218354 1607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:37.218587 1607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:37.218915 1607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:37.218969 1607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.219010 1607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:37.219038 1607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.224821 1607 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:42763
I20260927 18:47:37.225060 1735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:42763 every 8 connection(s)
I20260927 18:47:37.225219 1607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:37.227152 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1607
I20260927 18:47:37.227308 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:37.227512 15803 external_mini_cluster.cc:1428] Setting key 857640902978eae8daf37e77c2699a99
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
I20260927 18:47:37.236121 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.226020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:51957 (local address 127.15.110.254:36229)
0927 18:47:37.226082 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.226085 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.226286 (+ 201us) server_negotiation.cc:407] Connection header received
0927 18:47:37.226792 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.226794 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.226805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.226871 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:37.228146 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.228895 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.230205 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.230394 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.230451 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.233142 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:37.233144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:37.233144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:37.233152 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:37.234627 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.235093 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.235094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.235094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.235159 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.235457 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.235458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.235459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.235583 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:37.235653 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.235941 (+ 288us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.236028 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:47:37.236330 1738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.225983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36229 (local address 127.15.110.194:51957)
0927 18:47:37.226152 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.226195 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.226677 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.226945 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.226950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.227751 (+ 801us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:37.228039 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.228045 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.228999 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.229000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.230094 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.230097 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.230478 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.230480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.230642 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.231224 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:37.231232 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:37.233041 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:37.234762 (+ 1721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.234770 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.234774 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.234988 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.235272 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.235273 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.235273 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.235388 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.235656 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:37.235663 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:37.235811 (+ 148us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.235923 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.236085 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260927 18:47:37.236900 1736 heartbeater.cc:347] Connected to a master server at 127.15.110.254:36229
I20260927 18:47:37.237015 1736 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:37.237213 1736 heartbeater.cc:511] Master 127.15.110.254:36229 requested a full tablet report, sending...
I20260927 18:47:37.237629 1411 ts_manager.cc:194] Registered new tserver with Master: 47627083594e40b98acd6e25f8b37d66 (127.15.110.194:42763)
I20260927 18:47:37.238018 1411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:51957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:47:37.254195 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:39997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:37.335429 1741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:37.335618 1741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:37.335640 1741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:37.336814 1741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:37.336856 1741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:37.336884 1741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:37.338372 1741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:36229
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:37.338605 1741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:37.338858 1741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:37.343842 1749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.343930 1741 server_base.cc:1034] running on GCE node
W20260927 18:47:37.343886 1747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.343848 1746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.344262 1741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:37.344532 1741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:37.345750 1741 hybrid_clock.cc:648] HybridClock initialized: now 1790534857345724 us; error 35 us; skew 500 ppm
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:37.348428 1741 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:47:37.348896 1741 webserver.cc:460] Webserver started at http://127.15.110.195:38821/ using document root <none> and password file <none>
I20260927 18:47:37.349136 1741 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:37.349205 1741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:37.349359 1741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:37.350378 1741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "90e94dba97ca45f8a613b0e8548aaf79"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "f4880cf15b3626e03b22895be43a63fc"
server_key_iv: "a6ff0237cde6b53e3ccedcbe36d78e49"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.350739 1741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "90e94dba97ca45f8a613b0e8548aaf79"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "f4880cf15b3626e03b22895be43a63fc"
server_key_iv: "a6ff0237cde6b53e3ccedcbe36d78e49"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.352170 1741 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:37.352965 1756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.353127 1741 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:37.353231 1741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "90e94dba97ca45f8a613b0e8548aaf79"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "f4880cf15b3626e03b22895be43a63fc"
server_key_iv: "a6ff0237cde6b53e3ccedcbe36d78e49"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.353315 1741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:37.390890 1741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:37.393327 1741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:37.393494 1741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:37.393752 1741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:37.394054 1741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:37.394110 1741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.394153 1741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:37.394179 1741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.400022 1741 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40889
I20260927 18:47:37.400061 1869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40889 every 8 connection(s)
I20260927 18:47:37.400415 1741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:47:37 dist-test-slave-vs05 krb5kdc[1366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534857, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:37.409212 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1741
I20260927 18:47:37.409305 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:47:37.409497 15803 external_mini_cluster.cc:1428] Setting key dea226db711c0cca1108a371ce1049d6
I20260927 18:47:37.409545 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.401137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:35981 (local address 127.15.110.254:36229)
0927 18:47:37.401178 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.401185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.401452 (+ 267us) server_negotiation.cc:407] Connection header received
0927 18:47:37.401911 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.401912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.401922 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.401987 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:37.402893 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.403448 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.404460 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.404641 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.404688 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.406979 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:37.406980 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:37.406981 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:37.406987 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:37.408227 (+ 1240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.408580 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.408581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.408581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.408630 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.408893 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.408894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.408895 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.409005 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:37.409122 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.409372 (+ 250us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.409454 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:47:37.409761 1872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.401106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36229 (local address 127.15.110.195:35981)
0927 18:47:37.401294 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.401338 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.401812 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.402041 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.402044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.402590 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:37.402804 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.402808 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.403521 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.403521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.404377 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.404380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.404737 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.404738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.404844 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.405370 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:37.405378 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:37.406898 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:37.408303 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.408308 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.408310 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.408518 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.408716 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:37.408717 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:37.408717 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:37.408814 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:37.409113 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:37.409117 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:37.409226 (+ 109us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.409345 (+ 119us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.409496 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":94,"threads_started":1}
I20260927 18:47:37.410240 1870 heartbeater.cc:347] Connected to a master server at 127.15.110.254:36229
I20260927 18:47:37.410351 1870 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:37.410533 1870 heartbeater.cc:511] Master 127.15.110.254:36229 requested a full tablet report, sending...
I20260927 18:47:37.410956 1411 ts_manager.cc:194] Registered new tserver with Master: 90e94dba97ca45f8a613b0e8548aaf79 (127.15.110.195:40889)
I20260927 18:47:37.411414 1411 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:35981
I20260927 18:47:37.421407 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:37.428808 1450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.422753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52914 (local address 127.15.110.254:36229)
0927 18:47:37.422818 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.422820 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.422924 (+ 104us) server_negotiation.cc:407] Connection header received
0927 18:47:37.422962 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.422963 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.422973 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.423026 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:37.423642 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.424172 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.424863 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.425044 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.425097 (+ 53us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.426317 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:37.426318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:37.426319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:37.426324 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:37.427801 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.428155 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.428156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.428156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.428205 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:37.428379 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:37.428379 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:37.428379 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:37.428496 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:37.428572 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.428663 (+ 91us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.428743 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260927 18:47:37.430713 1411 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:37.432324 1411 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:37.439030 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.434851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50984 (local address 127.15.110.194:42763)
0927 18:47:37.435045 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.435049 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.435059 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:37.435093 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.435095 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.435110 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.435187 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:37.435432 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436491 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.437789 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.438543 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.438626 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.438687 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.438907 (+ 220us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.438949 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":85,"threads_started":1}
I20260927 18:47:37.439041 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.434537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42763 (local address 127.0.0.1:50984)
0927 18:47:37.434948 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.434957 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.434996 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.435289 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.435290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.435291 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:37.435338 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.435340 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436604 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.436605 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.437690 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.437692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.438629 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.438630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.438777 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.438794 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.438874 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.438943 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":46,"thread_start_us":67,"threads_started":1}
I20260927 18:47:37.439304 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.434641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59100 (local address 127.15.110.195:40889)
0927 18:47:37.434807 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.434813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.434826 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:47:37.434904 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.434906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.434924 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.435243 (+ 319us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:37.435317 (+ 74us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436441 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.437936 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.438604 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.438667 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.438710 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.439192 (+ 482us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.439233 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260927 18:47:37.439041 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.434537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40889 (local address 127.0.0.1:59100)
0927 18:47:37.434709 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.434749 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.434813 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.435053 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.435057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.435060 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:37.435139 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.435141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436540 (+ 1399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.436541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.437850 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.437853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.438687 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.438688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.438777 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.438794 (+ 17us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.438874 (+ 80us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.438943 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":114,"thread_start_us":55,"threads_started":1}
I20260927 18:47:37.439944 1881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.434791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:42953 (local address 127.0.0.1:55134)
0927 18:47:37.435065 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.435073 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.435107 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.435491 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.435492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.435495 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:37.435557 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.435559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436480 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.436481 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.437851 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.437853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.439672 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.439673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.439745 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.439750 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.439800 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.439847 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":62,"thread_start_us":68,"threads_started":1}
I20260927 18:47:37.439941 1670 tablet_service.cc:1467] Processing CreateTablet for tablet 50a973841dda4810bd160f1744f77fc2 (DEFAULT_TABLE table=test-table [id=5c077604c0b54edbb8598019abaff0d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:37.440140 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.435002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55134 (local address 127.15.110.193:42953)
0927 18:47:37.435211 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.435216 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.435225 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:47:37.435267 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.435268 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.435279 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.435369 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:37.435646 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.436364 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.438748 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.439518 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.439588 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.439638 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.440037 (+ 399us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.440084 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260927 18:47:37.440289 1670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a973841dda4810bd160f1744f77fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:37.440303 1804 tablet_service.cc:1467] Processing CreateTablet for tablet 50a973841dda4810bd160f1744f77fc2 (DEFAULT_TABLE table=test-table [id=5c077604c0b54edbb8598019abaff0d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:37.440585 1804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a973841dda4810bd160f1744f77fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:37.443327 1536 tablet_service.cc:1467] Processing CreateTablet for tablet 50a973841dda4810bd160f1744f77fc2 (DEFAULT_TABLE table=test-table [id=5c077604c0b54edbb8598019abaff0d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:37.443681 1536 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50a973841dda4810bd160f1744f77fc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:37.443876 1888 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Bootstrap starting.
I20260927 18:47:37.444537 1888 tablet_bootstrap.cc:654] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:37.445140 1888 log.cc:824] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:37.445559 1887 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Bootstrap starting.
I20260927 18:47:37.446075 1888 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: No bootstrap required, opened a new log
I20260927 18:47:37.446158 1888 ts_tablet_manager.cc:1397] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260927 18:47:37.446241 1887 tablet_bootstrap.cc:654] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:37.446768 1887 log.cc:824] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:37.447563 1891 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Bootstrap starting.
I20260927 18:47:37.447721 1887 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: No bootstrap required, opened a new log
I20260927 18:47:37.447813 1887 ts_tablet_manager.cc:1397] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260927 18:47:37.448175 1891 tablet_bootstrap.cc:654] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:37.448289 1888 raft_consensus.cc:348] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.448487 1888 raft_consensus.cc:374] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:37.448539 1888 raft_consensus.cc:729] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e94dba97ca45f8a613b0e8548aaf79, State: Initialized, Role: FOLLOWER
I20260927 18:47:37.448720 1888 consensus_queue.cc:260] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.448755 1891 log.cc:824] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:37.449007 1888 ts_tablet_manager.cc:1428] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:37.449156 1870 heartbeater.cc:503] Master 127.15.110.254:36229 was elected leader, sending a full tablet report...
I20260927 18:47:37.449465 1891 tablet_bootstrap.cc:492] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: No bootstrap required, opened a new log
I20260927 18:47:37.449553 1891 ts_tablet_manager.cc:1397] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:37.450102 1887 raft_consensus.cc:348] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.450234 1887 raft_consensus.cc:374] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:37.450258 1887 raft_consensus.cc:729] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47627083594e40b98acd6e25f8b37d66, State: Initialized, Role: FOLLOWER
I20260927 18:47:37.450383 1887 consensus_queue.cc:260] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.450644 1887 ts_tablet_manager.cc:1428] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:37.450654 1736 heartbeater.cc:503] Master 127.15.110.254:36229 was elected leader, sending a full tablet report...
I20260927 18:47:37.451536 1891 raft_consensus.cc:348] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.451710 1891 raft_consensus.cc:374] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:37.451766 1891 raft_consensus.cc:729] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 336e0b643c8d43c7a65158f9c0ca3001, State: Initialized, Role: FOLLOWER
I20260927 18:47:37.451877 1891 consensus_queue.cc:260] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.452131 1891 ts_tablet_manager.cc:1428] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:37.452152 1602 heartbeater.cc:503] Master 127.15.110.254:36229 was elected leader, sending a full tablet report...
W20260927 18:47:37.475812 1737 tablet.cc:2369] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:47:37.573421 1603 tablet.cc:2369] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:47:37.651041 1871 tablet.cc:2369] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:37.652652 1895 raft_consensus.cc:482] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:37.652771 1895 raft_consensus.cc:504] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.653105 1895 leader_election.cc:288] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e94dba97ca45f8a613b0e8548aaf79 (127.15.110.195:40889), 47627083594e40b98acd6e25f8b37d66 (127.15.110.194:42763)
I20260927 18:47:37.657454 1885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.653439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:46729 (local address 127.15.110.194:42763)
0927 18:47:37.653502 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.653505 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.653697 (+ 192us) server_negotiation.cc:407] Connection header received
0927 18:47:37.653796 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.653797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.653810 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.653876 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:37.654085 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.654872 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.656451 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.657058 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.657136 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.657145 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.657357 (+ 212us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.657400 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:47:37.657555 1897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.653346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40889 (local address 127.15.110.193:40015)
0927 18:47:37.653533 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.653545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.653725 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.653899 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.653900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.653902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:37.654003 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.654005 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.655002 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.655003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.656367 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.656369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.657178 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.657179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.657266 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.657273 (+ 7us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.657349 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.657406 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":107,"thread_start_us":63,"threads_started":1}
I20260927 18:47:37.657558 1896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.653355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42763 (local address 127.15.110.193:46729)
0927 18:47:37.653621 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.653631 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:37.653700 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:37.653900 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:37.653901 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:37.653902 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:37.654010 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.654012 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.655001 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.655003 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.656357 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:37.656361 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.657168 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.657169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:37.657266 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.657275 (+ 9us) client_negotiation.cc:769] Sending connection context
0927 18:47:37.657349 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:47:37.657406 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":17,"thread_start_us":56,"threads_started":1}
I20260927 18:47:37.657737 1883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.653460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:40015 (local address 127.15.110.195:40889)
0927 18:47:37.653511 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.653513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.653710 (+ 197us) server_negotiation.cc:407] Connection header received
0927 18:47:37.653797 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.653798 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.653808 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.653876 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:37.654085 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.654897 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.656450 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.657058 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.657136 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.657145 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.657602 (+ 457us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.657658 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260927 18:47:37.658069 1824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a973841dda4810bd160f1744f77fc2" candidate_uuid: "336e0b643c8d43c7a65158f9c0ca3001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e94dba97ca45f8a613b0e8548aaf79" is_pre_election: true
I20260927 18:47:37.658121 1690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a973841dda4810bd160f1744f77fc2" candidate_uuid: "336e0b643c8d43c7a65158f9c0ca3001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47627083594e40b98acd6e25f8b37d66" is_pre_election: true
I20260927 18:47:37.658268 1824 raft_consensus.cc:2454] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 336e0b643c8d43c7a65158f9c0ca3001 in term 0.
I20260927 18:47:37.658286 1690 raft_consensus.cc:2454] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 336e0b643c8d43c7a65158f9c0ca3001 in term 0.
I20260927 18:47:37.658495 1489 leader_election.cc:302] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336e0b643c8d43c7a65158f9c0ca3001, 90e94dba97ca45f8a613b0e8548aaf79; no voters:
I20260927 18:47:37.658635 1895 raft_consensus.cc:2781] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:37.658700 1895 raft_consensus.cc:482] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:37.658728 1895 raft_consensus.cc:3037] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:37.659456 1895 raft_consensus.cc:504] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.659617 1895 leader_election.cc:288] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [CANDIDATE]: Term 1 election: Requested vote from peers 90e94dba97ca45f8a613b0e8548aaf79 (127.15.110.195:40889), 47627083594e40b98acd6e25f8b37d66 (127.15.110.194:42763)
I20260927 18:47:37.659780 1690 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a973841dda4810bd160f1744f77fc2" candidate_uuid: "336e0b643c8d43c7a65158f9c0ca3001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47627083594e40b98acd6e25f8b37d66"
I20260927 18:47:37.659744 1824 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50a973841dda4810bd160f1744f77fc2" candidate_uuid: "336e0b643c8d43c7a65158f9c0ca3001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e94dba97ca45f8a613b0e8548aaf79"
I20260927 18:47:37.659855 1690 raft_consensus.cc:3037] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:37.659874 1824 raft_consensus.cc:3037] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:37.660733 1690 raft_consensus.cc:2454] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 336e0b643c8d43c7a65158f9c0ca3001 in term 1.
I20260927 18:47:37.660842 1824 raft_consensus.cc:2454] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 336e0b643c8d43c7a65158f9c0ca3001 in term 1.
I20260927 18:47:37.660920 1491 leader_election.cc:302] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 336e0b643c8d43c7a65158f9c0ca3001, 47627083594e40b98acd6e25f8b37d66; no voters:
I20260927 18:47:37.661021 1895 raft_consensus.cc:2781] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:37.661190 1895 raft_consensus.cc:686] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [term 1 LEADER]: Becoming Leader. State: Replica: 336e0b643c8d43c7a65158f9c0ca3001, State: Running, Role: LEADER
I20260927 18:47:37.661294 1895 consensus_queue.cc:237] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } }
I20260927 18:47:37.662102 1411 catalog_manager.cc:5495] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 reported cstate change: term changed from 0 to 1, leader changed from <none> to 336e0b643c8d43c7a65158f9c0ca3001 (127.15.110.193). New cstate: current_term: 1 leader_uuid: "336e0b643c8d43c7a65158f9c0ca3001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "336e0b643c8d43c7a65158f9c0ca3001" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 42953 } health_report { overall_health: HEALTHY } } }
I20260927 18:47:37.721674 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.719150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55146 (local address 127.15.110.193:42953)
0927 18:47:37.719293 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.719296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.719309 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:47:37.719358 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.719359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.719386 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.719509 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:37.719637 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.720171 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.720881 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.721090 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.721206 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.721328 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:37.721425 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:37.721501 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.721544 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.721555 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260927 18:47:37.723726 1690 raft_consensus.cc:1261] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 1 FOLLOWER]: Refusing update from remote peer 336e0b643c8d43c7a65158f9c0ca3001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:37.723747 1824 raft_consensus.cc:1261] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 1 FOLLOWER]: Refusing update from remote peer 336e0b643c8d43c7a65158f9c0ca3001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:37.724171 1895 consensus_queue.cc:1035] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "90e94dba97ca45f8a613b0e8548aaf79" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40889 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:37.724344 1898 consensus_queue.cc:1035] T 50a973841dda4810bd160f1744f77fc2 P 336e0b643c8d43c7a65158f9c0ca3001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47627083594e40b98acd6e25f8b37d66" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:37.727145 1902 mvcc.cc:205] Tried to move back new op lower bound from 7334030777233457152 to 7334030776981344256. Current Snapshot: MvccSnapshot[applied={T|T < 7334030777233457152 or (T in {7334030777233457152})}]
I20260927 18:47:37.729435 1411 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:37.729537 1411 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:37.730325 1411 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2163bc8265f74416a82a83a13e576771: Sending DeleteTablet for 3 replicas of tablet 50a973841dda4810bd160f1744f77fc2
I20260927 18:47:37.730654 1670 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a973841dda4810bd160f1744f77fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:37 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:50984
I20260927 18:47:37.730892 1910 tablet_replica.cc:330] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: stopping tablet replica
I20260927 18:47:37.730890 1804 tablet_service.cc:1514] Processing DeleteTablet for tablet 50a973841dda4810bd160f1744f77fc2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:37 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:59100
I20260927 18:47:37.730996 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1473
I20260927 18:47:37.731885 1910 raft_consensus.cc:2229] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:37.732059 1910 raft_consensus.cc:2258] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:37.732506 1910 ts_tablet_manager.cc:1905] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:37.733372 1911 tablet_replica.cc:330] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: stopping tablet replica
I20260927 18:47:37.733477 1911 raft_consensus.cc:2229] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:37.733587 1911 raft_consensus.cc:2258] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:37.734035 1911 ts_tablet_manager.cc:1905] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:37.734527 1910 ts_tablet_manager.cc:1918] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:37.734603 1910 log.cc:1196] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/50a973841dda4810bd160f1744f77fc2
I20260927 18:47:37.734882 1910 ts_tablet_manager.cc:1939] T 50a973841dda4810bd160f1744f77fc2 P 47627083594e40b98acd6e25f8b37d66: Deleting consensus metadata
I20260927 18:47:37.735354 1397 catalog_manager.cc:4841] TS 47627083594e40b98acd6e25f8b37d66 (127.15.110.194:42763): tablet 50a973841dda4810bd160f1744f77fc2 (table test-table [id=5c077604c0b54edbb8598019abaff0d8]) successfully deleted
I20260927 18:47:37.735781 1911 ts_tablet_manager.cc:1918] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:37.735837 1911 log.cc:1196] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/50a973841dda4810bd160f1744f77fc2
I20260927 18:47:37.736030 1911 ts_tablet_manager.cc:1939] T 50a973841dda4810bd160f1744f77fc2 P 90e94dba97ca45f8a613b0e8548aaf79: Deleting consensus metadata
I20260927 18:47:37.736382 1395 catalog_manager.cc:4841] TS 90e94dba97ca45f8a613b0e8548aaf79 (127.15.110.195:40889): tablet 50a973841dda4810bd160f1744f77fc2 (table test-table [id=5c077604c0b54edbb8598019abaff0d8]) successfully deleted
W20260927 18:47:37.736586 1395 connection.cc:535] client connection to 127.15.110.193:42953 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260927 18:47:37.736671 1395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260927 18:47:37.736994 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1607
I20260927 18:47:37.737171 1882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.737007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:42953 (local address 127.0.0.1:55158)
0927 18:47:37.737077 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:37.737133 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:42953: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260927 18:47:37.738644 1395 catalog_manager.cc:4584] TS 336e0b643c8d43c7a65158f9c0ca3001 (127.15.110.193:42953): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50a973841dda4810bd160f1744f77fc2: Network error: Client connection negotiation failed: client connection to 127.15.110.193:42953: connect: Connection refused (error 111)
I20260927 18:47:37.741677 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1741
I20260927 18:47:37.745903 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1379
2026-09-27T18:47:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2152 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-27T18:47:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:37Z Disabled control of system clock
I20260927 18:47:37.773420 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42003
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:42003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:37.853588 1916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:37.853802 1916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:37.853822 1916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:37.854883 1916 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:37.854916 1916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:37.854929 1916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:37.854941 1916 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:37.854954 1916 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:37.856300 1916 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:42003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:42003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:37.856494 1916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:37.856727 1916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:37.861608 1922 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.861656 1916 server_base.cc:1034] running on GCE node
W20260927 18:47:37.861599 1921 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:37.861599 1924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:37.862161 1916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:37.862416 1916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:37.863555 1916 hybrid_clock.cc:648] HybridClock initialized: now 1790534857863536 us; error 34 us; skew 500 ppm
I20260927 18:47:37.864702 1916 webserver.cc:460] Webserver started at http://127.15.110.254:43383/ using document root <none> and password file <none>
I20260927 18:47:37.864951 1916 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:37.865021 1916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:37.865172 1916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:37.866310 1916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "2329496b54de493ca6b20eb8ac28c849"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "5e97592dfc28c3c27a06689eccd4b384"
server_key_iv: "7e4f333a87ee47641f6ac0e8d4119f50"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.866714 1916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "2329496b54de493ca6b20eb8ac28c849"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "5e97592dfc28c3c27a06689eccd4b384"
server_key_iv: "7e4f333a87ee47641f6ac0e8d4119f50"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.868068 1916 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:37.868813 1930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:37.868980 1916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:37.869083 1916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "2329496b54de493ca6b20eb8ac28c849"
format_stamp: "Formatted at 2026-09-27 18:47:37 on dist-test-slave-vs05"
server_key: "5e97592dfc28c3c27a06689eccd4b384"
server_key_iv: "7e4f333a87ee47641f6ac0e8d4119f50"
server_key_version: "encryptionkey@0"
I20260927 18:47:37.869158 1916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:37.889830 1916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:37.890177 1916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:37.890349 1916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:37.894214 1916 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:42003
I20260927 18:47:37.894255 1982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:42003 every 8 connection(s)
I20260927 18:47:37.894631 1916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:37.895711 1983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:37.897508 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 1916
I20260927 18:47:37.897624 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:37.897811 15803 external_mini_cluster.cc:1428] Setting key 74bd7307d602e9e8502c42b4e6fe99ae
I20260927 18:47:37.898157 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: Bootstrap starting.
I20260927 18:47:37.898833 1983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:37.899132 1983 log.cc:824] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:37.899914 1983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: No bootstrap required, opened a new log
I20260927 18:47:37.902081 1983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } }
I20260927 18:47:37.902262 1983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:37.902302 1983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2329496b54de493ca6b20eb8ac28c849, State: Initialized, Role: FOLLOWER
I20260927 18:47:37.902441 1983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } }
I20260927 18:47:37.902522 1983 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:37.902555 1983 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:37.902607 1983 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:37.903259 1983 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } }
I20260927 18:47:37.903347 1983 leader_election.cc:302] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2329496b54de493ca6b20eb8ac28c849; no voters:
I20260927 18:47:37.903549 1983 leader_election.cc:288] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:37.903577 1988 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:37.903825 1988 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [term 1 LEADER]: Becoming Leader. State: Replica: 2329496b54de493ca6b20eb8ac28c849, State: Running, Role: LEADER
I20260927 18:47:37.903869 1983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:37.903936 1988 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } }
I20260927 18:47:37.904076 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:37.898368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49156 (local address 127.15.110.254:42003)
0927 18:47:37.898579 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:37.898586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:37.898611 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:47:37.899071 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:37.899077 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:37.899139 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:37.899315 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:37.899564 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.900506 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.901712 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:37.902128 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:37.902222 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:37.903086 (+ 864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:37.903090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:37.903091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:37.903344 (+ 253us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:37.903389 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:37.903404 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:37.903474 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:37.903582 (+ 108us) server_negotiation.cc:299] Negotiation successful
0927 18:47:37.903747 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260927 18:47:37.904505 1989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2329496b54de493ca6b20eb8ac28c849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } } }
I20260927 18:47:37.904583 1989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:37.904706 1989 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2329496b54de493ca6b20eb8ac28c849. Latest consensus state: current_term: 1 leader_uuid: "2329496b54de493ca6b20eb8ac28c849" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2329496b54de493ca6b20eb8ac28c849" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 42003 } } }
I20260927 18:47:37.904757 1989 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:37.905037 1997 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:37.905514 1997 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:37.907138 1997 catalog_manager.cc:1347] Generated new cluster ID: da8f3bea9c194c3582c6b3d93136696f
I20260927 18:47:37.907227 1997 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:37.923380 1997 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:37.923976 1997 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:37.926945 1997 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: Generated new TSK 0
I20260927 18:47:37.927119 1997 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:37.936672 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--builtin_ntp_servers=127.15.110.212:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:38.018989 2007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:38.019234 2007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:38.019271 2007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:38.020423 2007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:38.020458 2007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:38.020484 2007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:38.021801 2007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:38.022038 2007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:38.022323 2007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:38.028321 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:38.028321 2015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:38.028565 2007 server_base.cc:1034] running on GCE node
W20260927 18:47:38.028362 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:38.028936 2007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:38.029228 2007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:38.030387 2007 hybrid_clock.cc:648] HybridClock initialized: now 1790534858030366 us; error 39 us; skew 500 ppm
I20260927 18:47:38.031754 2007 webserver.cc:460] Webserver started at http://127.15.110.193:44907/ using document root <none> and password file <none>
I20260927 18:47:38.032018 2007 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:38.032063 2007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:38.032204 2007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:38.033254 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "419c6285eca193c5dbcb7f9179a84d53"
server_key_iv: "f016a17b739b7665a78e40c1ee517fcf"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.033615 2007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "419c6285eca193c5dbcb7f9179a84d53"
server_key_iv: "f016a17b739b7665a78e40c1ee517fcf"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.034960 2007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:47:38.035786 2021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.035943 2007 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:38.036026 2007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "419c6285eca193c5dbcb7f9179a84d53"
server_key_iv: "f016a17b739b7665a78e40c1ee517fcf"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.036099 2007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:38.049208 2007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:38.049535 2007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:38.049685 2007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:38.049921 2007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:38.050243 2007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:38.050297 2007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.050338 2007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:38.050374 2007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.056391 2007 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:41135
I20260927 18:47:38.056433 2134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:41135 every 8 connection(s)
I20260927 18:47:38.056771 2007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:38.061568 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2007
I20260927 18:47:38.061658 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:38.061877 15803 external_mini_cluster.cc:1428] Setting key 6bb648afc68bb9eff1e155bb53826779
I20260927 18:47:38.062215 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.057524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:57621 (local address 127.15.110.254:42003)
0927 18:47:38.057590 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.057592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.057829 (+ 237us) server_negotiation.cc:407] Connection header received
0927 18:47:38.058376 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.058377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.058388 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.058463 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:38.059242 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.059742 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.060731 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.060926 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.061003 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.061936 (+ 933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:38.061937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:38.061938 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:38.061943 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:38.061959 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:38.061963 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:38.062008 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.062103 (+ 95us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.062130 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:47:38.062435 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.057504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42003 (local address 127.15.110.193:57621)
0927 18:47:38.057706 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.057738 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.058265 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.058545 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.058548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.058557 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:38.059115 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.059120 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.059839 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.059841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.060617 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.060619 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.061010 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.061011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.061143 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.061782 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:38.061783 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:38.061819 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:38.061842 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:38.062008 (+ 166us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:38.062015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:38.062018 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.062106 (+ 88us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.062208 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260927 18:47:38.062899 2135 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42003
I20260927 18:47:38.062973 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--builtin_ntp_servers=127.15.110.212:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260927 18:47:38.063005 2135 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:38.063227 2135 heartbeater.cc:511] Master 127.15.110.254:42003 requested a full tablet report, sending...
I20260927 18:47:38.063750 1947 ts_manager.cc:194] Registered new tserver with Master: 32bef4b8b3c24375ba1507e6fd6aa3a1 (127.15.110.193:41135)
I20260927 18:47:38.064637 1947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.193:57621
W20260927 18:47:38.143458 2138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:38.143651 2138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:38.143671 2138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:38.144729 2138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:38.144761 2138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:38.144785 2138 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:38.146020 2138 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:38.146190 2138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:38.146413 2138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:38.151379 2146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:38.151386 2143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:38.151381 2144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:38.151396 2138 server_base.cc:1034] running on GCE node
I20260927 18:47:38.151851 2138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:38.152101 2138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:38.153290 2138 hybrid_clock.cc:648] HybridClock initialized: now 1790534858153276 us; error 32 us; skew 500 ppm
I20260927 18:47:38.154593 2138 webserver.cc:460] Webserver started at http://127.15.110.194:33177/ using document root <none> and password file <none>
I20260927 18:47:38.154834 2138 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:38.154881 2138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:38.155049 2138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:38.156179 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "f0c9053e0e384dfa94b95a2355a741f9"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "8f7cac9a842f056b028b6914eacc02a9"
server_key_iv: "59fc79475474cb6951f356e45341a639"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.156540 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "f0c9053e0e384dfa94b95a2355a741f9"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "8f7cac9a842f056b028b6914eacc02a9"
server_key_iv: "59fc79475474cb6951f356e45341a639"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.157823 2138 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:38.158501 2152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.158674 2138 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:38.158771 2138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "f0c9053e0e384dfa94b95a2355a741f9"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "8f7cac9a842f056b028b6914eacc02a9"
server_key_iv: "59fc79475474cb6951f356e45341a639"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.158855 2138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:38.191403 2138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:38.191754 2138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:38.191915 2138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:38.192162 2138 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:38.192515 2138 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:38.192572 2138 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.192608 2138 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:38.192652 2138 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.198216 2138 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:42623
I20260927 18:47:38.198251 2265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:42623 every 8 connection(s)
I20260927 18:47:38.198603 2138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:38.203812 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.199424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:59151 (local address 127.15.110.254:42003)
0927 18:47:38.199501 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.199503 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.199764 (+ 261us) server_negotiation.cc:407] Connection header received
0927 18:47:38.200217 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.200218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.200228 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.200284 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:38.200944 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.201427 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.202460 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.202634 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.202693 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.203578 (+ 885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:38.203579 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:38.203579 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:38.203584 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:38.203597 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:38.203601 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:38.203645 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.203727 (+ 82us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.203753 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:47:38.204005 2268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.199395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42003 (local address 127.15.110.194:59151)
0927 18:47:38.199629 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.199664 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.200136 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.200338 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.200341 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.200349 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:38.200866 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.200871 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.201507 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.201507 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.202374 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.202377 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.202727 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.202728 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.202847 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.203444 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:38.203444 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:38.203474 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:38.203497 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:38.203657 (+ 160us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:38.203661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:38.203664 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.203715 (+ 51us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.203794 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":106,"threads_started":1}
I20260927 18:47:38.204504 2266 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42003
I20260927 18:47:38.204605 2266 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:38.204784 2266 heartbeater.cc:511] Master 127.15.110.254:42003 requested a full tablet report, sending...
I20260927 18:47:38.205173 1947 ts_manager.cc:194] Registered new tserver with Master: f0c9053e0e384dfa94b95a2355a741f9 (127.15.110.194:42623)
I20260927 18:47:38.205549 1947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.194:59151
I20260927 18:47:38.208386 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2138
I20260927 18:47:38.208489 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:38.208678 15803 external_mini_cluster.cc:1428] Setting key a55686b0ae052f4128a1433ec0e62883
I20260927 18:47:38.209546 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--builtin_ntp_servers=127.15.110.212:45079
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:38.289312 2269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:38.289520 2269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:38.289543 2269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:38.290745 2269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:38.290789 2269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:38.290817 2269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:38.292212 2269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:45079
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:42003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:38.292471 2269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:38.292774 2269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:38.297680 2274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:38.297690 2277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:38.297731 2275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:38.298028 2269 server_base.cc:1034] running on GCE node
I20260927 18:47:38.298250 2269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:38.298485 2269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:38.299641 2269 hybrid_clock.cc:648] HybridClock initialized: now 1790534858299634 us; error 32 us; skew 500 ppm
I20260927 18:47:38.300823 2269 webserver.cc:460] Webserver started at http://127.15.110.195:41959/ using document root <none> and password file <none>
I20260927 18:47:38.301072 2269 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:38.301141 2269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:38.301271 2269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:38.302299 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "280a50371d104b649be7f9a677bdab3e"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "027ddfcd7ae76900c39859cbf8e38c1b"
server_key_iv: "2d08fac55f8b80ccaa298bb346f72fe1"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.302681 2269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "280a50371d104b649be7f9a677bdab3e"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "027ddfcd7ae76900c39859cbf8e38c1b"
server_key_iv: "2d08fac55f8b80ccaa298bb346f72fe1"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.304175 2269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:38.305073 2283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.305261 2269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:38.305349 2269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "280a50371d104b649be7f9a677bdab3e"
format_stamp: "Formatted at 2026-09-27 18:47:38 on dist-test-slave-vs05"
server_key: "027ddfcd7ae76900c39859cbf8e38c1b"
server_key_iv: "2d08fac55f8b80ccaa298bb346f72fe1"
server_key_version: "encryptionkey@0"
I20260927 18:47:38.305438 2269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:38.323925 2269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:38.324239 2269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:38.324401 2269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:38.324652 2269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:38.324971 2269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:38.325026 2269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.325060 2269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:38.325086 2269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:38.330768 2269 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:44209
I20260927 18:47:38.330819 2396 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:44209 every 8 connection(s)
I20260927 18:47:38.331132 2269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:47:38.334450 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2269
I20260927 18:47:38.334578 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:47:38.334810 15803 external_mini_cluster.cc:1428] Setting key 2857f5e750cd432ae9b273e1d2c9a631
I20260927 18:47:38.336417 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.331871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:43739 (local address 127.15.110.254:42003)
0927 18:47:38.331917 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.331919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.332128 (+ 209us) server_negotiation.cc:407] Connection header received
0927 18:47:38.332634 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.332636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.332647 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.332715 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:38.333377 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.333877 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.334907 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.335208 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.335284 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.336142 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:38.336144 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:38.336144 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:38.336149 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:38.336164 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:38.336168 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:38.336214 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.336306 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.336332 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:47:38.336627 2399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.331829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:42003 (local address 127.15.110.195:43739)
0927 18:47:38.332014 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.332056 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.332533 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.332788 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.332791 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.332799 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:38.333291 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.333295 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.333958 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.333959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.334811 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.334814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.335290 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.335291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.335430 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.335987 (+ 557us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:38.335987 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:38.336015 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:38.336038 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:38.336213 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:38.336218 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:38.336222 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.336309 (+ 87us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.336410 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":90,"threads_started":1}
I20260927 18:47:38.337100 2397 heartbeater.cc:347] Connected to a master server at 127.15.110.254:42003
I20260927 18:47:38.337232 2397 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:38.337415 2397 heartbeater.cc:511] Master 127.15.110.254:42003 requested a full tablet report, sending...
I20260927 18:47:38.337777 1947 ts_manager.cc:194] Registered new tserver with Master: 280a50371d104b649be7f9a677bdab3e (127.15.110.195:44209)
I20260927 18:47:38.338198 1947 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.195:43739
I20260927 18:47:38.346472 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:38.351155 1986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.348065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49170 (local address 127.15.110.254:42003)
0927 18:47:38.348135 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.348138 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.348225 (+ 87us) server_negotiation.cc:407] Connection header received
0927 18:47:38.348302 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.348303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.348314 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.348360 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:38.348548 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.349043 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.349834 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.350050 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.350131 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.350853 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:38.350854 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:38.350855 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:38.350859 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:38.350873 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:38.350877 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:38.350997 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.351063 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.351073 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:47:38.352701 1947 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:49170:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:38.354278 1947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:38.360577 2406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:44209 (local address 127.0.0.1:33498)
0927 18:47:38.356740 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.356779 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.356896 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.357203 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.357203 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.357205 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:38.357274 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.357275 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358709 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.358710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.359633 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.359635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360291 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360292 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.360363 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360376 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.360452 (+ 76us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.360503 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":4,"thread_start_us":67,"threads_started":1}
I20260927 18:47:38.360591 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.15.110.195:44209)
0927 18:47:38.356679 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.356684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.356867 (+ 183us) server_negotiation.cc:407] Connection header received
0927 18:47:38.357043 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.357046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.357063 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.357135 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:38.357613 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358639 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.359719 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360195 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360292 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360389 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.360465 (+ 76us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.360503 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260927 18:47:38.360900 2405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42623 (local address 127.0.0.1:33692)
0927 18:47:38.357014 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.357024 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.357066 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.357383 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.357383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.357385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:38.357418 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.357419 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358298 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.358299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.359651 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.359654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360642 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.360718 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360723 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.360775 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.360822 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":1,"thread_start_us":58,"threads_started":1}
I20260927 18:47:38.361168 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33692 (local address 127.15.110.194:42623)
0927 18:47:38.357225 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.357230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.357240 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:38.357298 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.357300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.357314 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.357386 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:38.357471 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358204 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.359749 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360471 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360541 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360608 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.361059 (+ 451us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.361095 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":83,"threads_started":1}
I20260927 18:47:38.361164 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39702 (local address 127.15.110.193:41135)
0927 18:47:38.356796 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.356800 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.356845 (+ 45us) server_negotiation.cc:407] Connection header received
0927 18:47:38.356995 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.356998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.357016 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.357096 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:38.357271 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358429 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.359764 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360471 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360524 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360604 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.361036 (+ 432us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.361083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":76,"threads_started":1}
I20260927 18:47:38.361514 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.356457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:41135 (local address 127.0.0.1:39702)
0927 18:47:38.356773 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.356784 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.356886 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.357108 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.357110 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.357113 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:38.357179 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.357181 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.358511 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.358512 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.359651 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.359654 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.360813 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.360814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.360864 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.360866 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.361389 (+ 523us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.361432 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":26,"thread_start_us":98,"threads_started":1}
I20260927 18:47:38.361334 2331 tablet_service.cc:1467] Processing CreateTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 (DEFAULT_TABLE table=test-table [id=76dc1a5c141642d7a1d172499032c62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:38.361765 2331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c5471ae3cd462e81f004ed8e98ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:38.362106 2069 tablet_service.cc:1467] Processing CreateTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 (DEFAULT_TABLE table=test-table [id=76dc1a5c141642d7a1d172499032c62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:38.362357 2069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c5471ae3cd462e81f004ed8e98ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:38.364524 2411 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Bootstrap starting.
I20260927 18:47:38.365015 2412 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: Bootstrap starting.
I20260927 18:47:38.365106 2200 tablet_service.cc:1467] Processing CreateTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 (DEFAULT_TABLE table=test-table [id=76dc1a5c141642d7a1d172499032c62a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:38.365401 2411 tablet_bootstrap.cc:654] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:38.365471 2200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3c5471ae3cd462e81f004ed8e98ab95. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:38.365679 2412 tablet_bootstrap.cc:654] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:38.365733 2411 log.cc:824] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:38.365986 2412 log.cc:824] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:38.366468 2411 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: No bootstrap required, opened a new log
I20260927 18:47:38.366550 2411 ts_tablet_manager.cc:1397] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:38.367828 2412 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: No bootstrap required, opened a new log
I20260927 18:47:38.367903 2412 ts_tablet_manager.cc:1397] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260927 18:47:38.368415 2415 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Bootstrap starting.
I20260927 18:47:38.368964 2411 raft_consensus.cc:348] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.369134 2415 tablet_bootstrap.cc:654] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:38.369145 2411 raft_consensus.cc:374] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:38.369184 2411 raft_consensus.cc:729] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280a50371d104b649be7f9a677bdab3e, State: Initialized, Role: FOLLOWER
I20260927 18:47:38.369343 2411 consensus_queue.cc:260] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.369449 2415 log.cc:824] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:38.369623 2411 ts_tablet_manager.cc:1428] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:47:38.369740 2397 heartbeater.cc:503] Master 127.15.110.254:42003 was elected leader, sending a full tablet report...
I20260927 18:47:38.370173 2415 tablet_bootstrap.cc:492] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: No bootstrap required, opened a new log
I20260927 18:47:38.370244 2415 ts_tablet_manager.cc:1397] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:38.370507 2412 raft_consensus.cc:348] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.370690 2412 raft_consensus.cc:374] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:38.370738 2412 raft_consensus.cc:729] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32bef4b8b3c24375ba1507e6fd6aa3a1, State: Initialized, Role: FOLLOWER
I20260927 18:47:38.370888 2412 consensus_queue.cc:260] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.371161 2412 ts_tablet_manager.cc:1428] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260927 18:47:38.371237 2135 heartbeater.cc:503] Master 127.15.110.254:42003 was elected leader, sending a full tablet report...
I20260927 18:47:38.372687 2415 raft_consensus.cc:348] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.372886 2415 raft_consensus.cc:374] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:38.372912 2415 raft_consensus.cc:729] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0c9053e0e384dfa94b95a2355a741f9, State: Initialized, Role: FOLLOWER
I20260927 18:47:38.372992 2415 consensus_queue.cc:260] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.373212 2415 ts_tablet_manager.cc:1428] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:38.373314 2266 heartbeater.cc:503] Master 127.15.110.254:42003 was elected leader, sending a full tablet report...
I20260927 18:47:38.377103 2418 raft_consensus.cc:482] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:38.377177 2418 raft_consensus.cc:504] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.377483 2418 leader_election.cc:288] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 280a50371d104b649be7f9a677bdab3e (127.15.110.195:44209), f0c9053e0e384dfa94b95a2355a741f9 (127.15.110.194:42623)
I20260927 18:47:38.380614 2137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.377680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:42623 (local address 127.15.110.193:35603)
0927 18:47:38.377830 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.377843 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.377901 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.378085 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.378086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.378088 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:38.378160 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.378162 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.378908 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.378913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.379748 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.379750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.380374 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.380376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.380446 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.380449 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.380501 (+ 52us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.380534 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":29}
I20260927 18:47:38.380645 2410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.377742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:35603 (local address 127.15.110.194:42623)
0927 18:47:38.377811 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.377814 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.377885 (+ 71us) server_negotiation.cc:407] Connection header received
0927 18:47:38.377982 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.377983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.377995 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.378067 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:38.378232 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.378800 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.379835 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.380284 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.380351 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.380387 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.380525 (+ 138us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.380571 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:47:38.380734 2407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.377869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:35911 (local address 127.15.110.195:44209)
0927 18:47:38.377987 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.377989 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.378089 (+ 100us) server_negotiation.cc:407] Connection header received
0927 18:47:38.378152 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.378153 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.378162 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.378217 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:38.378364 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.378791 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.379967 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.380306 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.380392 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.380400 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.380614 (+ 214us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.380660 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260927 18:47:38.380752 2420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.377690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:44209 (local address 127.15.110.193:35911)
0927 18:47:38.378013 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.378024 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:38.378086 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:38.378228 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:38.378229 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:38.378230 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:38.378298 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.378300 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.378869 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.378870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.379873 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:38.379876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.380422 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.380423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:38.380533 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.380538 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:47:38.380599 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:47:38.380637 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":123,"thread_start_us":120,"threads_started":1}
I20260927 18:47:38.380944 2220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c5471ae3cd462e81f004ed8e98ab95" candidate_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0c9053e0e384dfa94b95a2355a741f9" is_pre_election: true
I20260927 18:47:38.381088 2220 raft_consensus.cc:2454] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32bef4b8b3c24375ba1507e6fd6aa3a1 in term 0.
I20260927 18:47:38.381132 2351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c5471ae3cd462e81f004ed8e98ab95" candidate_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280a50371d104b649be7f9a677bdab3e" is_pre_election: true
I20260927 18:47:38.381253 2351 raft_consensus.cc:2454] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 32bef4b8b3c24375ba1507e6fd6aa3a1 in term 0.
I20260927 18:47:38.381347 2022 leader_election.cc:302] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 32bef4b8b3c24375ba1507e6fd6aa3a1, f0c9053e0e384dfa94b95a2355a741f9; no voters:
I20260927 18:47:38.381482 2418 raft_consensus.cc:2781] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:38.381569 2418 raft_consensus.cc:482] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:38.381603 2418 raft_consensus.cc:3037] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:38.382416 2418 raft_consensus.cc:504] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.382637 2418 leader_election.cc:288] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [CANDIDATE]: Term 1 election: Requested vote from peers 280a50371d104b649be7f9a677bdab3e (127.15.110.195:44209), f0c9053e0e384dfa94b95a2355a741f9 (127.15.110.194:42623)
I20260927 18:47:38.382725 2351 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c5471ae3cd462e81f004ed8e98ab95" candidate_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280a50371d104b649be7f9a677bdab3e"
I20260927 18:47:38.382716 2220 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3c5471ae3cd462e81f004ed8e98ab95" candidate_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0c9053e0e384dfa94b95a2355a741f9"
I20260927 18:47:38.382792 2351 raft_consensus.cc:3037] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:38.382858 2220 raft_consensus.cc:3037] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:38.383759 2351 raft_consensus.cc:2454] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32bef4b8b3c24375ba1507e6fd6aa3a1 in term 1.
I20260927 18:47:38.383811 2220 raft_consensus.cc:2454] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 32bef4b8b3c24375ba1507e6fd6aa3a1 in term 1.
I20260927 18:47:38.383982 2025 leader_election.cc:302] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 280a50371d104b649be7f9a677bdab3e, 32bef4b8b3c24375ba1507e6fd6aa3a1, f0c9053e0e384dfa94b95a2355a741f9; no voters:
I20260927 18:47:38.384102 2418 raft_consensus.cc:2781] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:38.384256 2418 raft_consensus.cc:686] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 1 LEADER]: Becoming Leader. State: Replica: 32bef4b8b3c24375ba1507e6fd6aa3a1, State: Running, Role: LEADER
I20260927 18:47:38.384363 2418 consensus_queue.cc:237] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } }
I20260927 18:47:38.385205 1947 catalog_manager.cc:5495] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 32bef4b8b3c24375ba1507e6fd6aa3a1 (127.15.110.193). New cstate: current_term: 1 leader_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "32bef4b8b3c24375ba1507e6fd6aa3a1" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 41135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 } health_report { overall_health: UNKNOWN } } }
I20260927 18:47:38.392424 2408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.389761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.15.110.193:41135)
0927 18:47:38.389923 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:38.389925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:38.389932 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:47:38.389958 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:38.389959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:38.389975 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:38.390035 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:38.390190 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.390658 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.391685 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:38.391868 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:38.391922 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:38.392110 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:38.392192 (+ 82us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:38.392265 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:38.392365 (+ 100us) server_negotiation.cc:299] Negotiation successful
0927 18:47:38.392375 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260927 18:47:38.394107 2220 raft_consensus.cc:1261] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 1 FOLLOWER]: Refusing update from remote peer 32bef4b8b3c24375ba1507e6fd6aa3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:38.394109 2351 raft_consensus.cc:1261] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 1 FOLLOWER]: Refusing update from remote peer 32bef4b8b3c24375ba1507e6fd6aa3a1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:38.394440 2418 consensus_queue.cc:1035] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0c9053e0e384dfa94b95a2355a741f9" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 42623 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:38.394582 2421 consensus_queue.cc:1035] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280a50371d104b649be7f9a677bdab3e" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 44209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:38.397092 2424 mvcc.cc:205] Tried to move back new op lower bound from 7334030779979644928 to 7334030779943038976. Current Snapshot: MvccSnapshot[applied={T|T < 7334030779979644928}]
I20260927 18:47:38.397217 2426 mvcc.cc:205] Tried to move back new op lower bound from 7334030779979644928 to 7334030779943038976. Current Snapshot: MvccSnapshot[applied={T|T < 7334030779979644928}]
I20260927 18:47:38.399541 1947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:38.399617 1947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:49170:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:38.400251 1947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2329496b54de493ca6b20eb8ac28c849: Sending DeleteTablet for 3 replicas of tablet a3c5471ae3cd462e81f004ed8e98ab95
I20260927 18:47:38.400558 2331 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:38 UTC) from {username='slave'} at 127.0.0.1:33498
I20260927 18:47:38.400647 2069 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:38 UTC) from {username='slave'} at 127.0.0.1:39702
I20260927 18:47:38.400683 2200 tablet_service.cc:1514] Processing DeleteTablet for tablet a3c5471ae3cd462e81f004ed8e98ab95 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:38 UTC) from {username='slave'} at 127.0.0.1:33692
I20260927 18:47:38.400892 2436 tablet_replica.cc:330] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1: stopping tablet replica
I20260927 18:47:38.400988 2436 raft_consensus.cc:2229] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:47:38.401113 2436 raft_consensus.cc:2258] T a3c5471ae3cd462e81f004ed8e98ab95 P 32bef4b8b3c24375ba1507e6fd6aa3a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:38.401144 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2007
I20260927 18:47:38.401667 2435 tablet_replica.cc:330] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: stopping tablet replica
I20260927 18:47:38.401760 2435 raft_consensus.cc:2229] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:38.401854 2435 raft_consensus.cc:2258] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:38.402287 2435 ts_tablet_manager.cc:1905] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:38.402671 2434 tablet_replica.cc:330] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: stopping tablet replica
I20260927 18:47:38.402765 2434 raft_consensus.cc:2229] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:38.402854 2434 raft_consensus.cc:2258] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:38.403297 2434 ts_tablet_manager.cc:1905] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:38.404426 2435 ts_tablet_manager.cc:1918] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:38.404497 2435 log.cc:1196] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/a3c5471ae3cd462e81f004ed8e98ab95
I20260927 18:47:38.404835 2435 ts_tablet_manager.cc:1939] T a3c5471ae3cd462e81f004ed8e98ab95 P f0c9053e0e384dfa94b95a2355a741f9: Deleting consensus metadata
I20260927 18:47:38.404960 2434 ts_tablet_manager.cc:1918] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:38.405015 2434 log.cc:1196] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/a3c5471ae3cd462e81f004ed8e98ab95
I20260927 18:47:38.405311 2434 ts_tablet_manager.cc:1939] T a3c5471ae3cd462e81f004ed8e98ab95 P 280a50371d104b649be7f9a677bdab3e: Deleting consensus metadata
I20260927 18:47:38.405290 1931 catalog_manager.cc:4841] TS f0c9053e0e384dfa94b95a2355a741f9 (127.15.110.194:42623): tablet a3c5471ae3cd462e81f004ed8e98ab95 (table test-table [id=76dc1a5c141642d7a1d172499032c62a]) successfully deleted
I20260927 18:47:38.405881 1934 catalog_manager.cc:4841] TS 280a50371d104b649be7f9a677bdab3e (127.15.110.195:44209): tablet a3c5471ae3cd462e81f004ed8e98ab95 (table test-table [id=76dc1a5c141642d7a1d172499032c62a]) successfully deleted
W20260927 18:47:38.407437 1931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:41135 (error 108)
I20260927 18:47:38.407684 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2138
I20260927 18:47:38.408262 2409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:38.407785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:41135 (local address 127.0.0.1:39722)
0927 18:47:38.408162 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:38.408224 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:41135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356}
W20260927 18:47:38.408418 1931 catalog_manager.cc:4584] TS 32bef4b8b3c24375ba1507e6fd6aa3a1 (127.15.110.193:41135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3c5471ae3cd462e81f004ed8e98ab95: Network error: Client connection negotiation failed: client connection to 127.15.110.193:41135: connect: Connection refused (error 111)
I20260927 18:47:38.412257 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2269
I20260927 18:47:38.416595 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 1916
2026-09-27T18:47:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (669 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:38 dist-test-slave-vs05 krb5kdc[2440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:38 dist-test-slave-vs05 krb5kdc[2440](info): set up 2 sockets
Sep 27 18:47:38 dist-test-slave-vs05 krb5kdc[2440](info): commencing operation
krb5kdc: starting...
W20260927 18:47:39.455168 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:39 dist-test-slave-vs05 krb5kdc[2440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:47:39.574532 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43505
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:43505
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:39.655279 2453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:39.655529 2453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:39.655567 2453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:39.656780 2453 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:39.656849 2453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:39.656874 2453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:39.656898 2453 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:39.656925 2453 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:39.658393 2453 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:43505
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:39.658674 2453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:39.658914 2453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:39.663847 2458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:39.663865 2459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:39.663883 2461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:39.664050 2453 server_base.cc:1034] running on GCE node
I20260927 18:47:39.664309 2453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:39.664592 2453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:39.665772 2453 hybrid_clock.cc:648] HybridClock initialized: now 1790534859665789 us; error 60 us; skew 500 ppm
Sep 27 18:47:39 dist-test-slave-vs05 krb5kdc[2440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534859, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:39.668320 2453 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:39.668833 2453 webserver.cc:460] Webserver started at http://127.15.110.254:35661/ using document root <none> and password file <none>
I20260927 18:47:39.669060 2453 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:39.669128 2453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:39.669282 2453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:39.670315 2453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "67ad16e21a564392ae912714a898b717"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "64aecc7a813825115464d65fc3d3e10b"
server_key_iv: "e5c8c0374fff058f75fef6b20dd729ae"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.670678 2453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "67ad16e21a564392ae912714a898b717"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "64aecc7a813825115464d65fc3d3e10b"
server_key_iv: "e5c8c0374fff058f75fef6b20dd729ae"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.672245 2453 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:39.673089 2468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:39.673346 2453 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:39.673460 2453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "67ad16e21a564392ae912714a898b717"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "64aecc7a813825115464d65fc3d3e10b"
server_key_iv: "e5c8c0374fff058f75fef6b20dd729ae"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.673535 2453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:39.686128 2453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:39.688246 2453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:39.688411 2453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:39.692538 2453 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:43505
I20260927 18:47:39.692596 2520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:43505 every 8 connection(s)
I20260927 18:47:39.692970 2453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:39.693892 2521 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:39.696439 2521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: Bootstrap starting.
I20260927 18:47:39.697002 2521 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:39.697270 2521 log.cc:824] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:39.697985 2521 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: No bootstrap required, opened a new log
I20260927 18:47:39.699056 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2453
I20260927 18:47:39.699221 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:39.699424 15803 external_mini_cluster.cc:1428] Setting key 4e84e650ab120f3b7e4efc75e9f9cb21
I20260927 18:47:39.699857 2521 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } }
I20260927 18:47:39.700043 2521 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:39.700081 2521 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67ad16e21a564392ae912714a898b717, State: Initialized, Role: FOLLOWER
I20260927 18:47:39.700227 2521 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } }
I20260927 18:47:39.700330 2521 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:39.700372 2521 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:39.700426 2521 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:39.701277 2521 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } }
I20260927 18:47:39.701400 2521 leader_election.cc:302] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67ad16e21a564392ae912714a898b717; no voters:
I20260927 18:47:39.701583 2521 leader_election.cc:288] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:39.701619 2526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:39.701756 2526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [term 1 LEADER]: Becoming Leader. State: Replica: 67ad16e21a564392ae912714a898b717, State: Running, Role: LEADER
I20260927 18:47:39.701853 2521 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:39.701838 2526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } }
I20260927 18:47:39.702155 2526 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67ad16e21a564392ae912714a898b717. Latest consensus state: current_term: 1 leader_uuid: "67ad16e21a564392ae912714a898b717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } } }
I20260927 18:47:39.702224 2526 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:39.702163 2528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67ad16e21a564392ae912714a898b717" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67ad16e21a564392ae912714a898b717" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43505 } } }
I20260927 18:47:39.702533 2528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:39.703048 2531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:39.703558 2531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:39.705307 2531 catalog_manager.cc:1347] Generated new cluster ID: 49316fb3518849d9bd02083b6f2498df
I20260927 18:47:39.705364 2531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:39 dist-test-slave-vs05 krb5kdc[2440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:39.710739 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:39.699936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51374 (local address 127.15.110.254:43505)
0927 18:47:39.700144 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:39.700153 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:39.700187 (+ 34us) server_negotiation.cc:407] Connection header received
0927 18:47:39.700298 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:39.700304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:39.700380 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:39.700511 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:39.701139 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.702435 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.703527 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.704108 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.704227 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:39.707102 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:39.707106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:39.707106 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:39.707118 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:39.709124 (+ 2006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:39.709530 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:39.709532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:39.709533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:39.709641 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:39.709868 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:39.709868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:39.709869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:39.710146 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:39.710288 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:39.710423 (+ 135us) server_negotiation.cc:299] Negotiation successful
0927 18:47:39.710549 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":92,"threads_started":1}
I20260927 18:47:39.731412 2531 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:39.732002 2531 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:39.738579 2531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: Generated new TSK 0
I20260927 18:47:39.738803 2531 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.15.110.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
Entry for principal kudu/127.15.110.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab.
I20260927 18:47:39.770175 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:39.852438 2547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:39.852656 2547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:39.852684 2547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:39.853835 2547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:39.853879 2547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:39.853914 2547 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:39.855506 2547 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.193.keytab
--principal=kudu/127.15.110.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:39.855764 2547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:39.856014 2547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:39.861528 2552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:39.861567 2555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:39.861572 2553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:39.861743 2547 server_base.cc:1034] running on GCE node
I20260927 18:47:39.861922 2547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:39.862118 2547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:39.863251 2547 hybrid_clock.cc:648] HybridClock initialized: now 1790534859863228 us; error 27 us; skew 500 ppm
Sep 27 18:47:39 dist-test-slave-vs05 krb5kdc[2440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534859, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:39.865773 2547 init.cc:377] Logged in from keytab as kudu/127.15.110.193@KRBTEST.COM (short username kudu)
I20260927 18:47:39.866148 2547 webserver.cc:460] Webserver started at http://127.15.110.193:35545/ using document root <none> and password file <none>
I20260927 18:47:39.866379 2547 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:39.866457 2547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:39.866606 2547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:39.867713 2547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "b3b6abcae2b54774893ef7829388e4b0"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "de63a1883de6f481ff61bb0615757b48"
server_key_iv: "e4daa3d0754a48a23918d2c0c0202823"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.868075 2547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "b3b6abcae2b54774893ef7829388e4b0"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "de63a1883de6f481ff61bb0615757b48"
server_key_iv: "e4daa3d0754a48a23918d2c0c0202823"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.869405 2547 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:39.870138 2562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:39.870311 2547 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:39.870409 2547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "b3b6abcae2b54774893ef7829388e4b0"
format_stamp: "Formatted at 2026-09-27 18:47:39 on dist-test-slave-vs05"
server_key: "de63a1883de6f481ff61bb0615757b48"
server_key_iv: "e4daa3d0754a48a23918d2c0c0202823"
server_key_version: "encryptionkey@0"
I20260927 18:47:39.870491 2547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:39.890856 2547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:39.893307 2547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:39.893477 2547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:39.893734 2547 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:39.894047 2547 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:39.894101 2547 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:39.894146 2547 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:39.894169 2547 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:39.900048 2547 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:34965
I20260927 18:47:39.900092 2675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:34965 every 8 connection(s)
I20260927 18:47:39.900419 2547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:39.905241 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2547
I20260927 18:47:39.905372 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:39.905645 15803 external_mini_cluster.cc:1428] Setting key f4498ba217ccdeabd54b912c3f5f5162
Sep 27 18:47:39 dist-test-slave-vs05 krb5kdc[2440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534859, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.193@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:39.910612 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:39.901182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:44227 (local address 127.15.110.254:43505)
0927 18:47:39.901252 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:39.901254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:39.901491 (+ 237us) server_negotiation.cc:407] Connection header received
0927 18:47:39.902097 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:39.902099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:39.902114 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:39.902179 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:39.903263 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.903859 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.904855 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.905086 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.905159 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:39.907639 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:39.907641 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:39.907641 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:39.907649 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:39.909119 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:39.909647 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:39.909647 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:39.909648 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:39.909700 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:39.909963 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:39.909965 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:39.909965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:39.910081 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:39.910158 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:39.910431 (+ 273us) server_negotiation.cc:299] Negotiation successful
0927 18:47:39.910517 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:47:39.910812 2678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:39.901158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:43505 (local address 127.15.110.193:44227)
0927 18:47:39.901355 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:39.901398 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:39.901891 (+ 493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:39.902262 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:39.902266 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:39.902959 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:39.903158 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:39.903164 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.903944 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.903945 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:39.904752 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:39.904754 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:39.905185 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:39.905187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:39.905345 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:39.906031 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:39.906041 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:39.907531 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:39.909213 (+ 1682us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:39.909221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:39.909223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:39.909562 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:39.909772 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:39.909773 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:39.909774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:39.909891 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:39.910149 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:39.910156 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:39.910294 (+ 138us) client_negotiation.cc:769] Sending connection context
0927 18:47:39.910406 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 18:47:39.910570 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":108,"threads_started":1}
I20260927 18:47:39.911396 2676 heartbeater.cc:347] Connected to a master server at 127.15.110.254:43505
I20260927 18:47:39.911480 2676 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:39.911695 2676 heartbeater.cc:511] Master 127.15.110.254:43505 requested a full tablet report, sending...
I20260927 18:47:39.912281 2485 ts_manager.cc:194] Registered new tserver with Master: b3b6abcae2b54774893ef7829388e4b0 (127.15.110.193:34965)
I20260927 18:47:39.913240 2485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.193@KRBTEST.COM'} at 127.15.110.193:44227
WARNING: no policy specified for kudu/127.15.110.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
Entry for principal kudu/127.15.110.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab.
I20260927 18:47:39.931993 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:40.013237 2681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:40.013437 2681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:40.013459 2681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:40.014647 2681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:40.014691 2681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:40.014725 2681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:40.016108 2681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.194.keytab
--principal=kudu/127.15.110.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:40.016355 2681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:40.016551 2681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:40.021466 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.021536 2681 server_base.cc:1034] running on GCE node
W20260927 18:47:40.021476 2689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:40.021543 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.021867 2681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:40.022099 2681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:40.023262 2681 hybrid_clock.cc:648] HybridClock initialized: now 1790534860023246 us; error 27 us; skew 500 ppm
Sep 27 18:47:40 dist-test-slave-vs05 krb5kdc[2440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:40.025600 2681 init.cc:377] Logged in from keytab as kudu/127.15.110.194@KRBTEST.COM (short username kudu)
I20260927 18:47:40.026027 2681 webserver.cc:460] Webserver started at http://127.15.110.194:32825/ using document root <none> and password file <none>
I20260927 18:47:40.026257 2681 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:40.026304 2681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:40.026470 2681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:40.027554 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "e896373cae234020805321608ece97b0"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "76130e937f07713d3e388542d9ce77e3"
server_key_iv: "babce4fcc856113f0cb03ef28fb3952a"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.027917 2681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "e896373cae234020805321608ece97b0"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "76130e937f07713d3e388542d9ce77e3"
server_key_iv: "babce4fcc856113f0cb03ef28fb3952a"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.029206 2681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:40.029920 2696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.030134 2681 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:40.030221 2681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "e896373cae234020805321608ece97b0"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "76130e937f07713d3e388542d9ce77e3"
server_key_iv: "babce4fcc856113f0cb03ef28fb3952a"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.030282 2681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:40.044162 2681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:40.046063 2681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:40.046180 2681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:40.046370 2681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:40.046623 2681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:40.046656 2681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.046689 2681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:40.046708 2681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.052805 2681 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:44811
I20260927 18:47:40.052856 2809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:44811 every 8 connection(s)
I20260927 18:47:40.053170 2681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:40.056869 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2681
I20260927 18:47:40.056993 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:40.057196 15803 external_mini_cluster.cc:1428] Setting key 5c3924b9552d5b171412af68f3e45dc9
Sep 27 18:47:40 dist-test-slave-vs05 krb5kdc[2440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.194@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:40.063949 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.053903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:46625 (local address 127.15.110.254:43505)
0927 18:47:40.053951 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.053953 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.054194 (+ 241us) server_negotiation.cc:407] Connection header received
0927 18:47:40.054829 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.054830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.054842 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.054916 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:40.056042 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.056590 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.058013 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.058223 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.058302 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.060905 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:40.060906 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:40.060906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:40.060912 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:40.062387 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.062931 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.062932 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.062932 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.062985 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.063316 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.063317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.063317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.063425 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:40.063478 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.063818 (+ 340us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.063884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:47:40.064206 2812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.053875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:43505 (local address 127.15.110.194:46625)
0927 18:47:40.054067 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.054105 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.054642 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.054994 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.054999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.055681 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:40.055917 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.055922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.056691 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.056692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.057907 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.057910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.058320 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.058321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.058460 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.059000 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:40.059009 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:40.060794 (+ 1785us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:40.062473 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:40.062481 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:40.062484 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:40.062837 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:40.063058 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:40.063059 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:40.063059 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:40.063243 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:40.063494 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:40.063501 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:40.063667 (+ 166us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.063778 (+ 111us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.063957 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260927 18:47:40.064752 2810 heartbeater.cc:347] Connected to a master server at 127.15.110.254:43505
I20260927 18:47:40.064843 2810 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:40.065038 2810 heartbeater.cc:511] Master 127.15.110.254:43505 requested a full tablet report, sending...
I20260927 18:47:40.065438 2485 ts_manager.cc:194] Registered new tserver with Master: e896373cae234020805321608ece97b0 (127.15.110.194:44811)
WARNING: no policy specified for kudu/127.15.110.195@KRBTEST.COM; defaulting to no policy
I20260927 18:47:40.065860 2485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.194@KRBTEST.COM'} at 127.15.110.194:46625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
Entry for principal kudu/127.15.110.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab.
I20260927 18:47:40.082755 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:44439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:40.162176 2815 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:40.162366 2815 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:40.162389 2815 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:40.163571 2815 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:40.163612 2815 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:40.163638 2815 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:40.165102 2815 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.195.keytab
--principal=kudu/127.15.110.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:43505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2815
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:40.165343 2815 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:40.165573 2815 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:40.170583 2823 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:40.170593 2820 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:40.170641 2821 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.170938 2815 server_base.cc:1034] running on GCE node
I20260927 18:47:40.171224 2815 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:40.171480 2815 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:40.172627 2815 hybrid_clock.cc:648] HybridClock initialized: now 1790534860172604 us; error 35 us; skew 500 ppm
Sep 27 18:47:40 dist-test-slave-vs05 krb5kdc[2440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:40.175251 2815 init.cc:377] Logged in from keytab as kudu/127.15.110.195@KRBTEST.COM (short username kudu)
I20260927 18:47:40.175747 2815 webserver.cc:460] Webserver started at http://127.15.110.195:44007/ using document root <none> and password file <none>
I20260927 18:47:40.176004 2815 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:40.176050 2815 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:40.176204 2815 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:40.177367 2815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "271609fce0b349ad9f5cd03eb082289d"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "53765fbc45c6fbefc292820752c4244f"
server_key_iv: "46ca073ca18c2207ef3bef9daf52cc67"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.177719 2815 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "271609fce0b349ad9f5cd03eb082289d"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "53765fbc45c6fbefc292820752c4244f"
server_key_iv: "46ca073ca18c2207ef3bef9daf52cc67"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.179086 2815 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:40.179877 2830 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.180054 2815 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:40.180143 2815 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "271609fce0b349ad9f5cd03eb082289d"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "53765fbc45c6fbefc292820752c4244f"
server_key_iv: "46ca073ca18c2207ef3bef9daf52cc67"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.180224 2815 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:40.198139 2815 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:40.200122 2815 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:40.200291 2815 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:40.200528 2815 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:40.200843 2815 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:40.200905 2815 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.200948 2815 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:40.200976 2815 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.207307 2815 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:43083
I20260927 18:47:40.207378 2943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:43083 every 8 connection(s)
I20260927 18:47:40.207715 2815 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
Sep 27 18:47:40 dist-test-slave-vs05 krb5kdc[2440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534860, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.195@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:40.216732 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.208458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:36173 (local address 127.15.110.254:43505)
0927 18:47:40.208530 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.208532 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.208757 (+ 225us) server_negotiation.cc:407] Connection header received
0927 18:47:40.209197 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.209198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.209208 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.209267 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:40.210148 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.210746 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.211787 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.211969 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.212025 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.214009 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:40.214010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:40.214010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:40.214017 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:40.215517 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.215925 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.215926 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.215926 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.215978 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.216213 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.216213 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.216214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.216315 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:40.216378 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.216603 (+ 225us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.216661 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:47:40.216892 2946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.208428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:43505 (local address 127.15.110.195:36173)
0927 18:47:40.208589 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.208631 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.209111 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.209318 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.209321 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.209882 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:40.210069 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.210074 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.210847 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.210848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.211718 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.211720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.212041 (+ 321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.212042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.212140 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.212601 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:47:40.212609 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:40.213927 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:40.215613 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:40.215618 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:40.215620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:40.215851 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:40.216038 (+ 187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:47:40.216039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:47:40.216039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:47:40.216136 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:47:40.216380 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:40.216384 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:40.216493 (+ 109us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.216575 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.216692 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":88,"threads_started":1}
I20260927 18:47:40.217435 2944 heartbeater.cc:347] Connected to a master server at 127.15.110.254:43505
I20260927 18:47:40.217516 2944 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:40.217525 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2815
I20260927 18:47:40.217610 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:47:40.217790 15803 external_mini_cluster.cc:1428] Setting key 795c75966fecd1c5e8b8a82d78ee0e65
I20260927 18:47:40.217782 2944 heartbeater.cc:511] Master 127.15.110.254:43505 requested a full tablet report, sending...
I20260927 18:47:40.218256 2485 ts_manager.cc:194] Registered new tserver with Master: 271609fce0b349ad9f5cd03eb082289d (127.15.110.195:43083)
I20260927 18:47:40.218664 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:40.218680 2485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.15.110.195@KRBTEST.COM'} at 127.15.110.195:36173
I20260927 18:47:40.225368 2525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.219953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42360 (local address 127.15.110.254:43505)
0927 18:47:40.220010 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.220012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.220095 (+ 83us) server_negotiation.cc:407] Connection header received
0927 18:47:40.220164 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.220165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.220173 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.220221 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:40.220684 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.221119 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.221829 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.222000 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.222051 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.223312 (+ 1261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:40.223313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:40.223313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:40.223319 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:40.224462 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.224772 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.224773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.224773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.224817 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:40.224971 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:40.224971 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:40.224972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:40.225091 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:40.225161 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.225249 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.225299 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:47:40.226864 2485 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42360:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:40.228422 2485 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:40.234230 2954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44811 (local address 127.0.0.1:39190)
0927 18:47:40.230845 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.230866 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.230928 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.231165 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.231166 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.231168 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:40.231246 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.231248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232155 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.232157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.233122 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.233125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.233941 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.233942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.234016 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.234029 (+ 13us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.234103 (+ 74us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.234149 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":62,"threads_started":1}
I20260927 18:47:40.234249 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39190 (local address 127.15.110.194:44811)
0927 18:47:40.230883 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.230887 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.230913 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:47:40.230994 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.230996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.231014 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.231091 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:40.231330 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232055 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.233219 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.233864 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.233937 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.233994 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.234126 (+ 132us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.234168 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260927 18:47:40.235013 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51278 (local address 127.15.110.195:43083)
0927 18:47:40.230805 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.230810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.231054 (+ 244us) server_negotiation.cc:407] Connection header received
0927 18:47:40.231151 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.231153 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.231170 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.231288 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:40.231480 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232500 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.233816 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.234593 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.234660 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.234710 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.234894 (+ 184us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.234930 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":82,"threads_started":1}
I20260927 18:47:40.235028 2953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:43083 (local address 127.0.0.1:51278)
0927 18:47:40.231001 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.231010 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.231080 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.231357 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.231357 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.231359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:40.231400 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.231403 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232602 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.232603 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.233711 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.233715 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.234749 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.234750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.234805 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.234808 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.234878 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.234921 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":7,"thread_start_us":51,"threads_started":1}
I20260927 18:47:40.235227 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.15.110.193:34965)
0927 18:47:40.230843 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.230847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.230859 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:47:40.230934 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.230936 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.230951 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.231023 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:40.231216 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232380 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.233847 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.234619 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.234689 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.234746 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.235071 (+ 325us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.235115 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":75,"threads_started":1}
I20260927 18:47:40.235034 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.230424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:34965 (local address 127.0.0.1:54922)
0927 18:47:40.230748 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.230790 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.230856 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.231044 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.231046 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.231049 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:40.231105 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.231107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.232489 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.232490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.233768 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.233770 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.234782 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.234783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.234845 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.234849 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.234898 (+ 49us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.234937 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":21,"thread_start_us":62,"threads_started":1}
I20260927 18:47:40.235139 2744 tablet_service.cc:1467] Processing CreateTablet for tablet c0af16c854624563890f4c6fab756b22 (DEFAULT_TABLE table=test-table [id=8c3a2dbf6cad4b2c83c2db8aa7966b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:40.235570 2744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0af16c854624563890f4c6fab756b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:40.235975 2610 tablet_service.cc:1467] Processing CreateTablet for tablet c0af16c854624563890f4c6fab756b22 (DEFAULT_TABLE table=test-table [id=8c3a2dbf6cad4b2c83c2db8aa7966b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:40.236205 2610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0af16c854624563890f4c6fab756b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:40.238025 2958 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Bootstrap starting.
I20260927 18:47:40.238672 2878 tablet_service.cc:1467] Processing CreateTablet for tablet c0af16c854624563890f4c6fab756b22 (DEFAULT_TABLE table=test-table [id=8c3a2dbf6cad4b2c83c2db8aa7966b05]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:40.238744 2959 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Bootstrap starting.
I20260927 18:47:40.238972 2958 tablet_bootstrap.cc:654] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:40.238965 2878 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0af16c854624563890f4c6fab756b22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:40.239527 2959 tablet_bootstrap.cc:654] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:40.239559 2958 log.cc:824] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:40.240080 2959 log.cc:824] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:40.240468 2958 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: No bootstrap required, opened a new log
I20260927 18:47:40.240513 2958 ts_tablet_manager.cc:1397] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260927 18:47:40.240854 2959 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: No bootstrap required, opened a new log
I20260927 18:47:40.240908 2959 ts_tablet_manager.cc:1397] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260927 18:47:40.241576 2962 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Bootstrap starting.
I20260927 18:47:40.242200 2962 tablet_bootstrap.cc:654] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:40.242527 2958 raft_consensus.cc:348] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.242751 2962 log.cc:824] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:40.242786 2958 raft_consensus.cc:374] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:40.242833 2958 raft_consensus.cc:729] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e896373cae234020805321608ece97b0, State: Initialized, Role: FOLLOWER
I20260927 18:47:40.242976 2958 consensus_queue.cc:260] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.243438 2810 heartbeater.cc:503] Master 127.15.110.254:43505 was elected leader, sending a full tablet report...
I20260927 18:47:40.243319 2959 raft_consensus.cc:348] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.243491 2959 raft_consensus.cc:374] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:40.243525 2959 raft_consensus.cc:729] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3b6abcae2b54774893ef7829388e4b0, State: Initialized, Role: FOLLOWER
I20260927 18:47:40.243637 2959 consensus_queue.cc:260] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.243706 2962 tablet_bootstrap.cc:492] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: No bootstrap required, opened a new log
I20260927 18:47:40.243727 2958 ts_tablet_manager.cc:1428] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:40.243767 2962 ts_tablet_manager.cc:1397] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:40.243901 2959 ts_tablet_manager.cc:1428] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260927 18:47:40.244014 2676 heartbeater.cc:503] Master 127.15.110.254:43505 was elected leader, sending a full tablet report...
I20260927 18:47:40.245489 2962 raft_consensus.cc:348] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.245630 2962 raft_consensus.cc:374] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:40.245674 2962 raft_consensus.cc:729] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 271609fce0b349ad9f5cd03eb082289d, State: Initialized, Role: FOLLOWER
I20260927 18:47:40.245777 2962 consensus_queue.cc:260] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.246026 2962 ts_tablet_manager.cc:1428] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260927 18:47:40.246057 2944 heartbeater.cc:503] Master 127.15.110.254:43505 was elected leader, sending a full tablet report...
W20260927 18:47:40.303915 2811 tablet.cc:2369] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:47:40.401213 2677 tablet.cc:2369] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260927 18:47:40.458278 2945 tablet.cc:2369] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:40.513842 2966 raft_consensus.cc:482] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:40.513960 2966 raft_consensus.cc:504] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.514335 2966 leader_election.cc:288] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b3b6abcae2b54774893ef7829388e4b0 (127.15.110.193:34965), e896373cae234020805321608ece97b0 (127.15.110.194:44811)
I20260927 18:47:40.518220 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.514542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:36217 (local address 127.15.110.193:34965)
0927 18:47:40.514586 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.514588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.514605 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:47:40.514670 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.514671 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.514685 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.514756 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:40.514998 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.515796 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517270 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.517819 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517897 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.517905 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.518121 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.518157 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:47:40.518220 2957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.514746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:38629 (local address 127.15.110.194:44811)
0927 18:47:40.514804 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.514806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.514814 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:47:40.514838 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.514839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.514861 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.514905 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:40.515087 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.515799 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517267 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.517820 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517897 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.517905 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.518108 (+ 203us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.518148 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260927 18:47:40.518236 2946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.514490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:34965 (local address 127.15.110.195:36217)
0927 18:47:40.514539 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.514552 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.514601 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.514811 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.514812 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.514814 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:40.514907 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.514909 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.515887 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.515888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.517167 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.517170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.517921 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.518019 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.518020 (+ 1us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.518092 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.518142 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":74}
I20260927 18:47:40.518257 2967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.514514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44811 (local address 127.15.110.195:38629)
0927 18:47:40.514669 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.514680 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.514731 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.514935 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.514936 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.514938 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:40.515013 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.515018 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.515893 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.515894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.517167 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.517170 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.517921 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.517923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.518016 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.518020 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.518092 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.518142 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":36,"thread_start_us":100,"threads_started":1}
I20260927 18:47:40.518630 2764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0af16c854624563890f4c6fab756b22" candidate_uuid: "271609fce0b349ad9f5cd03eb082289d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e896373cae234020805321608ece97b0" is_pre_election: true
I20260927 18:47:40.518803 2764 raft_consensus.cc:2454] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 271609fce0b349ad9f5cd03eb082289d in term 0.
I20260927 18:47:40.518951 2630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0af16c854624563890f4c6fab756b22" candidate_uuid: "271609fce0b349ad9f5cd03eb082289d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3b6abcae2b54774893ef7829388e4b0" is_pre_election: true
I20260927 18:47:40.519024 2831 leader_election.cc:302] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271609fce0b349ad9f5cd03eb082289d, e896373cae234020805321608ece97b0; no voters:
I20260927 18:47:40.519098 2630 raft_consensus.cc:2454] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 271609fce0b349ad9f5cd03eb082289d in term 0.
I20260927 18:47:40.519138 2966 raft_consensus.cc:2781] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:40.519227 2966 raft_consensus.cc:482] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:40.519253 2966 raft_consensus.cc:3037] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:40.519969 2966 raft_consensus.cc:504] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.520110 2966 leader_election.cc:288] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [CANDIDATE]: Term 1 election: Requested vote from peers b3b6abcae2b54774893ef7829388e4b0 (127.15.110.193:34965), e896373cae234020805321608ece97b0 (127.15.110.194:44811)
I20260927 18:47:40.520319 2764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0af16c854624563890f4c6fab756b22" candidate_uuid: "271609fce0b349ad9f5cd03eb082289d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e896373cae234020805321608ece97b0"
I20260927 18:47:40.520346 2630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0af16c854624563890f4c6fab756b22" candidate_uuid: "271609fce0b349ad9f5cd03eb082289d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3b6abcae2b54774893ef7829388e4b0"
I20260927 18:47:40.520411 2630 raft_consensus.cc:3037] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:40.520412 2764 raft_consensus.cc:3037] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:40.521308 2630 raft_consensus.cc:2454] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 271609fce0b349ad9f5cd03eb082289d in term 1.
I20260927 18:47:40.521391 2764 raft_consensus.cc:2454] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 271609fce0b349ad9f5cd03eb082289d in term 1.
I20260927 18:47:40.521505 2833 leader_election.cc:302] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 271609fce0b349ad9f5cd03eb082289d, b3b6abcae2b54774893ef7829388e4b0; no voters:
I20260927 18:47:40.521644 2966 raft_consensus.cc:2781] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:40.521802 2966 raft_consensus.cc:686] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 1 LEADER]: Becoming Leader. State: Replica: 271609fce0b349ad9f5cd03eb082289d, State: Running, Role: LEADER
I20260927 18:47:40.521903 2966 consensus_queue.cc:237] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } }
I20260927 18:47:40.522783 2485 catalog_manager.cc:5495] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d reported cstate change: term changed from 0 to 1, leader changed from <none> to 271609fce0b349ad9f5cd03eb082289d (127.15.110.195). New cstate: current_term: 1 leader_uuid: "271609fce0b349ad9f5cd03eb082289d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "271609fce0b349ad9f5cd03eb082289d" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 43083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 } health_report { overall_health: UNKNOWN } } }
I20260927 18:47:40.587469 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.584988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.15.110.195:43083)
0927 18:47:40.585052 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.585056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.585068 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:47:40.585105 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.585106 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.585124 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.585212 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:40.585400 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.585859 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.586619 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.586794 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.586842 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.587020 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:40.587119 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:40.587195 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.587365 (+ 170us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.587386 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:47:40.589524 2630 raft_consensus.cc:1261] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 1 FOLLOWER]: Refusing update from remote peer 271609fce0b349ad9f5cd03eb082289d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:40.589524 2764 raft_consensus.cc:1261] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 1 FOLLOWER]: Refusing update from remote peer 271609fce0b349ad9f5cd03eb082289d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:40.589918 2966 consensus_queue.cc:1035] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [LEADER]: Connected to new peer: Peer: permanent_uuid: "e896373cae234020805321608ece97b0" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:40.590080 2968 consensus_queue.cc:1035] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3b6abcae2b54774893ef7829388e4b0" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 34965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:40.592655 2972 mvcc.cc:205] Tried to move back new op lower bound from 7334030788971433984 to 7334030788698333184. Current Snapshot: MvccSnapshot[applied={T|T < 7334030788971433984}]
I20260927 18:47:40.593730 2973 mvcc.cc:205] Tried to move back new op lower bound from 7334030788971433984 to 7334030788698333184. Current Snapshot: MvccSnapshot[applied={T|T < 7334030788971433984 or (T in {7334030788971433984})}]
I20260927 18:47:40.594831 2484 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:40.594935 2484 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42360:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:40.595758 2484 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67ad16e21a564392ae912714a898b717: Sending DeleteTablet for 3 replicas of tablet c0af16c854624563890f4c6fab756b22
I20260927 18:47:40.596076 2610 tablet_service.cc:1514] Processing DeleteTablet for tablet c0af16c854624563890f4c6fab756b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:54922
I20260927 18:47:40.596086 2878 tablet_service.cc:1514] Processing DeleteTablet for tablet c0af16c854624563890f4c6fab756b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:51278
I20260927 18:47:40.596114 2744 tablet_service.cc:1514] Processing DeleteTablet for tablet c0af16c854624563890f4c6fab756b22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:40 UTC) from {username='kudu', principal='kudu/127.15.110.254@KRBTEST.COM'} at 127.0.0.1:39190
I20260927 18:47:40.596288 2982 tablet_replica.cc:330] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0: stopping tablet replica
I20260927 18:47:40.596383 2982 raft_consensus.cc:2229] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:40.596499 2982 raft_consensus.cc:2258] T c0af16c854624563890f4c6fab756b22 P b3b6abcae2b54774893ef7829388e4b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:40.596544 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2547
I20260927 18:47:40.600142 2981 tablet_replica.cc:330] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: stopping tablet replica
I20260927 18:47:40.600268 2981 raft_consensus.cc:2229] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:47:40.600369 2981 raft_consensus.cc:2258] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:40.600520 2983 tablet_replica.cc:330] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: stopping tablet replica
I20260927 18:47:40.600622 2983 raft_consensus.cc:2229] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:40.600736 2983 raft_consensus.cc:2258] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:40.601231 2983 ts_tablet_manager.cc:1905] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:40.602108 2981 ts_tablet_manager.cc:1905] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260927 18:47:40.603092 2471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:34965 (error 108)
I20260927 18:47:40.603411 2983 ts_tablet_manager.cc:1918] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:40.603487 2983 log.cc:1196] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/c0af16c854624563890f4c6fab756b22
I20260927 18:47:40.603739 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2681
I20260927 18:47:40.603780 2983 ts_tablet_manager.cc:1939] T c0af16c854624563890f4c6fab756b22 P e896373cae234020805321608ece97b0: Deleting consensus metadata
I20260927 18:47:40.604209 2981 ts_tablet_manager.cc:1918] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:40.604314 2981 log.cc:1196] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/c0af16c854624563890f4c6fab756b22
I20260927 18:47:40.604592 2981 ts_tablet_manager.cc:1939] T c0af16c854624563890f4c6fab756b22 P 271609fce0b349ad9f5cd03eb082289d: Deleting consensus metadata
I20260927 18:47:40.605242 2472 catalog_manager.cc:4841] TS 271609fce0b349ad9f5cd03eb082289d (127.15.110.195:43083): tablet c0af16c854624563890f4c6fab756b22 (table test-table [id=8c3a2dbf6cad4b2c83c2db8aa7966b05]) successfully deleted
I20260927 18:47:40.606739 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.606583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:34965 (local address 127.0.0.1:54928)
0927 18:47:40.606653 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.606703 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:34965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260927 18:47:40.606871 2471 catalog_manager.cc:4584] TS b3b6abcae2b54774893ef7829388e4b0 (127.15.110.193:34965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0af16c854624563890f4c6fab756b22: Network error: Client connection negotiation failed: client connection to 127.15.110.193:34965: connect: Connection refused (error 111)
I20260927 18:47:40.609862 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2815
I20260927 18:47:40.609886 2952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.609788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44811 (local address 127.0.0.1:39198)
0927 18:47:40.609828 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.609861 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.194:44811: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260927 18:47:40.614912 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2453
2026-09-27T18:47:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2199 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-27T18:47:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:40Z Disabled control of system clock
I20260927 18:47:40.642325 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44809
--webserver_interface=127.15.110.254
--webserver_port=0
--builtin_ntp_servers=127.15.110.212:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:44809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:40.723083 2988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:40.723321 2988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:40.723342 2988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:40.724433 2988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:40.724467 2988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:40.724479 2988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:40.724493 2988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:40.724504 2988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:40.725768 2988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:44809
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:44809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:40.725953 2988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:40.726263 2988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:40.731557 2994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:40.731590 2993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.731746 2988 server_base.cc:1034] running on GCE node
W20260927 18:47:40.731552 2996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.732059 2988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:40.732389 2988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:40.733567 2988 hybrid_clock.cc:648] HybridClock initialized: now 1790534860733553 us; error 45 us; skew 500 ppm
I20260927 18:47:40.734952 2988 webserver.cc:460] Webserver started at http://127.15.110.254:45211/ using document root <none> and password file <none>
I20260927 18:47:40.735289 2988 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:40.735368 2988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:40.735526 2988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:40.736658 2988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "c7e7d24647664fc5806671c43b8a4bdc"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "995af4ac21089089bce8236f1e3f3b47"
server_key_iv: "9e263714b656340b14bb7535a599a2d7"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.737056 2988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "c7e7d24647664fc5806671c43b8a4bdc"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "995af4ac21089089bce8236f1e3f3b47"
server_key_iv: "9e263714b656340b14bb7535a599a2d7"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.738482 2988 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:40.739419 3002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.739629 2988 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:40.739778 2988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "c7e7d24647664fc5806671c43b8a4bdc"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "995af4ac21089089bce8236f1e3f3b47"
server_key_iv: "9e263714b656340b14bb7535a599a2d7"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.739862 2988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:40.757721 2988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:40.758067 2988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:40.758226 2988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:40.762100 2988 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:44809
I20260927 18:47:40.762140 3054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:44809 every 8 connection(s)
I20260927 18:47:40.762483 2988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:40.763597 3055 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:40.766486 3055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: Bootstrap starting.
I20260927 18:47:40.766846 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 2988
I20260927 18:47:40.767005 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:40.767228 3055 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:40.767243 15803 external_mini_cluster.cc:1428] Setting key b370de860b22baa396c209453415116d
I20260927 18:47:40.767594 3055 log.cc:824] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:40.768523 3055 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: No bootstrap required, opened a new log
I20260927 18:47:40.770619 3055 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } }
I20260927 18:47:40.770788 3055 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:40.770833 3055 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7e7d24647664fc5806671c43b8a4bdc, State: Initialized, Role: FOLLOWER
I20260927 18:47:40.770968 3055 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } }
I20260927 18:47:40.771070 3055 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:40.771116 3055 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:40.771165 3055 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:40.771914 3055 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } }
I20260927 18:47:40.772035 3055 leader_election.cc:302] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7e7d24647664fc5806671c43b8a4bdc; no voters:
I20260927 18:47:40.772223 3055 leader_election.cc:288] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:40.772395 3060 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:40.772521 3055 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:40.772555 3060 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [term 1 LEADER]: Becoming Leader. State: Replica: c7e7d24647664fc5806671c43b8a4bdc, State: Running, Role: LEADER
I20260927 18:47:40.772653 3060 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } }
I20260927 18:47:40.772943 3060 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7e7d24647664fc5806671c43b8a4bdc. Latest consensus state: current_term: 1 leader_uuid: "c7e7d24647664fc5806671c43b8a4bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } } }
I20260927 18:47:40.773028 3060 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:40.772953 3062 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7e7d24647664fc5806671c43b8a4bdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7e7d24647664fc5806671c43b8a4bdc" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 44809 } } }
I20260927 18:47:40.773104 3062 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:40.773293 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.767764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.15.110.254:44809)
0927 18:47:40.767935 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.767943 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.767969 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:47:40.768392 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.768398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.768451 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.768571 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:40.768846 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.769709 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.770857 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.771416 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.771504 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.772320 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:40.772325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:40.772325 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:40.772530 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:40.772573 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:40.772588 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:40.772908 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.772943 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.773035 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260927 18:47:40.773440 3067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:40.774158 3067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:40.775782 3067 catalog_manager.cc:1347] Generated new cluster ID: b5ebdbb1a4f140bf93eacd7c4dd81f34
I20260927 18:47:40.775835 3067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:40.782840 3067 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:40.783519 3067 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:40.788029 3067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: Generated new TSK 0
I20260927 18:47:40.788213 3067 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:40.797510 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.193:0
--local_ip_for_outbound_sockets=127.15.110.193
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--builtin_ntp_servers=127.15.110.212:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:40.880733 3079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:40.880940 3079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:40.880963 3079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:40.882205 3079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:40.882246 3079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:40.882272 3079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.193
I20260927 18:47:40.883692 3079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.15.110.193
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.193
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:40.883939 3079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:40.884248 3079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:40.890311 3085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.890448 3079 server_base.cc:1034] running on GCE node
W20260927 18:47:40.890319 3084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:40.890301 3087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:40.890836 3079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:40.891085 3079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:40.892241 3079 hybrid_clock.cc:648] HybridClock initialized: now 1790534860892221 us; error 36 us; skew 500 ppm
I20260927 18:47:40.893538 3079 webserver.cc:460] Webserver started at http://127.15.110.193:39337/ using document root <none> and password file <none>
I20260927 18:47:40.893801 3079 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:40.893870 3079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:40.894011 3079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:40.895076 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data/instance:
uuid: "39c5eba051654ca99b0ccb7934d0a635"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "8d8cce72f25cc17748c2cb7479e0404d"
server_key_iv: "b0cbeeec361ff770a65198c61e272ba2"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.895485 3079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance:
uuid: "39c5eba051654ca99b0ccb7934d0a635"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "8d8cce72f25cc17748c2cb7479e0404d"
server_key_iv: "b0cbeeec361ff770a65198c61e272ba2"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.896795 3079 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:40.897596 3093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.897775 3079 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:40.897866 3079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal
uuid: "39c5eba051654ca99b0ccb7934d0a635"
format_stamp: "Formatted at 2026-09-27 18:47:40 on dist-test-slave-vs05"
server_key: "8d8cce72f25cc17748c2cb7479e0404d"
server_key_iv: "b0cbeeec361ff770a65198c61e272ba2"
server_key_version: "encryptionkey@0"
I20260927 18:47:40.897943 3079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:40.918211 3079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:40.918560 3079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:40.918711 3079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:40.918959 3079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:40.919349 3079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:40.919406 3079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.919442 3079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:40.919469 3079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:40.925117 3079 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.193:43427
I20260927 18:47:40.925240 3206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.193:43427 every 8 connection(s)
I20260927 18:47:40.925525 3079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/data/info.pb
I20260927 18:47:40.931125 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.926295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.193:60401 (local address 127.15.110.254:44809)
0927 18:47:40.926386 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:40.926389 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:40.926631 (+ 242us) server_negotiation.cc:407] Connection header received
0927 18:47:40.927224 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:40.927226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:40.927239 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:40.927292 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:40.928087 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.928618 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.929643 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.929854 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.929942 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.930813 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:40.930814 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:40.930815 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:40.930820 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:40.930835 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:40.930839 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:40.930916 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:40.931013 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 18:47:40.931042 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260927 18:47:40.931361 3209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:40.926221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44809 (local address 127.15.110.193:60401)
0927 18:47:40.926499 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:40.926536 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:40.927075 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:40.927383 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:40.927387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:40.927402 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:40.927989 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.927995 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.928728 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.928729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.929539 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:40.929541 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:40.929973 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:40.929975 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:40.930131 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:40.930620 (+ 489us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:40.930620 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:40.930648 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:40.930672 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:40.930922 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:40.930928 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:40.930932 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:40.931017 (+ 85us) client_negotiation.cc:240] Negotiation successful
0927 18:47:40.931121 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
I20260927 18:47:40.931830 3207 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44809
I20260927 18:47:40.931922 3207 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:40.932116 3207 heartbeater.cc:511] Master 127.15.110.254:44809 requested a full tablet report, sending...
I20260927 18:47:40.932582 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3079
I20260927 18:47:40.932647 3019 ts_manager.cc:194] Registered new tserver with Master: 39c5eba051654ca99b0ccb7934d0a635 (127.15.110.193:43427)
I20260927 18:47:40.932687 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-0/wal/instance
I20260927 18:47:40.932886 15803 external_mini_cluster.cc:1428] Setting key a7a6e458d876eb5d62e8e15e53ca6a67
I20260927 18:47:40.933396 3019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.193:60401
I20260927 18:47:40.933899 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.194:0
--local_ip_for_outbound_sockets=127.15.110.194
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--builtin_ntp_servers=127.15.110.212:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:41.019568 3210 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:41.019831 3210 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:41.019862 3210 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:41.021394 3210 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:41.021435 3210 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:41.021459 3210 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.194
I20260927 18:47:41.022751 3210 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.15.110.194
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3210
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.194
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:41.022940 3210 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:41.023252 3210 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:41.028090 3216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:41.028085 3218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:41.028188 3210 server_base.cc:1034] running on GCE node
W20260927 18:47:41.028237 3215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:41.028522 3210 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:41.028761 3210 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:41.029927 3210 hybrid_clock.cc:648] HybridClock initialized: now 1790534861029926 us; error 41 us; skew 500 ppm
I20260927 18:47:41.031139 3210 webserver.cc:460] Webserver started at http://127.15.110.194:46735/ using document root <none> and password file <none>
I20260927 18:47:41.031390 3210 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:41.031457 3210 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:41.031602 3210 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:41.032694 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data/instance:
uuid: "8ac463f217284abf944d9c46dd081c70"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "ccdd2aed16f2efa5cebf4827a954d002"
server_key_iv: "ef8dd601734c8e92c1f54a28e59ed492"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.033057 3210 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance:
uuid: "8ac463f217284abf944d9c46dd081c70"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "ccdd2aed16f2efa5cebf4827a954d002"
server_key_iv: "ef8dd601734c8e92c1f54a28e59ed492"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.034366 3210 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:41.035041 3224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.035233 3210 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:41.035328 3210 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal
uuid: "8ac463f217284abf944d9c46dd081c70"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "ccdd2aed16f2efa5cebf4827a954d002"
server_key_iv: "ef8dd601734c8e92c1f54a28e59ed492"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.035405 3210 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:41.063108 3210 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:41.063447 3210 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:41.063598 3210 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:41.063829 3210 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:41.064163 3210 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:41.064216 3210 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.064252 3210 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:41.064278 3210 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.069911 3210 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.194:44959
I20260927 18:47:41.069965 3337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.194:44959 every 8 connection(s)
I20260927 18:47:41.070271 3210 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/data/info.pb
I20260927 18:47:41.075596 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.070959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:57433 (local address 127.15.110.254:44809)
0927 18:47:41.071024 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.071026 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.071231 (+ 205us) server_negotiation.cc:407] Connection header received
0927 18:47:41.071769 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.071771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.071783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.071834 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:41.072523 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.073105 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.074150 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.074346 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.074425 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.075301 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:41.075302 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:41.075302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:41.075307 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:41.075321 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:41.075325 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:41.075398 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.075490 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.075517 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:47:41.075790 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.070920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44809 (local address 127.15.110.194:57433)
0927 18:47:41.071099 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.071139 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.071656 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.071923 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.071927 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.071937 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:41.072432 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.072437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.073189 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.073190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.074066 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.074069 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.074437 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.074438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.074569 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.075132 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:41.075133 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:41.075162 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:41.075207 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:41.075401 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:41.075407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:41.075410 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.075492 (+ 82us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.075593 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":71,"threads_started":1}
I20260927 18:47:41.076231 3338 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44809
I20260927 18:47:41.076314 3338 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:41.076462 3338 heartbeater.cc:511] Master 127.15.110.254:44809 requested a full tablet report, sending...
I20260927 18:47:41.076771 3019 ts_manager.cc:194] Registered new tserver with Master: 8ac463f217284abf944d9c46dd081c70 (127.15.110.194:44959)
I20260927 18:47:41.077111 3019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.194:57433
I20260927 18:47:41.078754 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3210
I20260927 18:47:41.078838 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal/instance
I20260927 18:47:41.079005 15803 external_mini_cluster.cc:1428] Setting key e6f700c73cd8c58fe495620d837efa28
I20260927 18:47:41.079836 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.15.110.195:0
--local_ip_for_outbound_sockets=127.15.110.195
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--builtin_ntp_servers=127.15.110.212:44993
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260927 18:47:41.159690 3341 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:41.159922 3341 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:41.159956 3341 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:41.161077 3341 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:41.161142 3341 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:41.161190 3341 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.15.110.195
I20260927 18:47:41.162528 3341 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:44993
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.15.110.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.15.110.195
--webserver_port=0
--tserver_master_addrs=127.15.110.254:44809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3341
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.15.110.195
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:41.162807 3341 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:41.163093 3341 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:41.168045 3346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:41.168073 3349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:41.168064 3347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:41.168238 3341 server_base.cc:1034] running on GCE node
I20260927 18:47:41.168529 3341 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:41.168756 3341 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:41.169914 3341 hybrid_clock.cc:648] HybridClock initialized: now 1790534861169904 us; error 29 us; skew 500 ppm
I20260927 18:47:41.171162 3341 webserver.cc:460] Webserver started at http://127.15.110.195:44575/ using document root <none> and password file <none>
I20260927 18:47:41.171452 3341 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:41.171521 3341 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:41.171657 3341 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:41.172750 3341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data/instance:
uuid: "814545dc07864ca68cb280a9a8cfa4a1"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "165aa9bfa3efb518974f002dd63f23b9"
server_key_iv: "2da9f894859f4df2a9d349e523a76ff6"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.173127 3341 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance:
uuid: "814545dc07864ca68cb280a9a8cfa4a1"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "165aa9bfa3efb518974f002dd63f23b9"
server_key_iv: "2da9f894859f4df2a9d349e523a76ff6"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.174479 3341 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:41.175140 3355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.175318 3341 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:41.175428 3341 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal
uuid: "814545dc07864ca68cb280a9a8cfa4a1"
format_stamp: "Formatted at 2026-09-27 18:47:41 on dist-test-slave-vs05"
server_key: "165aa9bfa3efb518974f002dd63f23b9"
server_key_iv: "2da9f894859f4df2a9d349e523a76ff6"
server_key_version: "encryptionkey@0"
I20260927 18:47:41.175500 3341 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:41.194185 3341 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:41.194490 3341 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:41.194633 3341 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:41.194846 3341 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260927 18:47:41.195235 3341 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260927 18:47:41.195286 3341 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.195327 3341 ts_tablet_manager.cc:610] Registered 0 tablets
I20260927 18:47:41.195363 3341 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:41.201004 3341 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.195:40153
I20260927 18:47:41.201076 3468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.195:40153 every 8 connection(s)
I20260927 18:47:41.201455 3341 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/data/info.pb
I20260927 18:47:41.204145 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3341
I20260927 18:47:41.204283 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal/instance
I20260927 18:47:41.204475 15803 external_mini_cluster.cc:1428] Setting key 3c70839589c59f32bd652a07fc150993
I20260927 18:47:41.206887 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.202201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.195:40199 (local address 127.15.110.254:44809)
0927 18:47:41.202247 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.202249 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.202519 (+ 270us) server_negotiation.cc:407] Connection header received
0927 18:47:41.203062 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.203063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.203075 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.203134 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:41.203826 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.204449 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.205449 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.205658 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.205737 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.206634 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:41.206635 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:41.206635 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:41.206640 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:41.206654 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:41.206658 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:41.206705 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.206778 (+ 73us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.206803 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:47:41.207099 3471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.202181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:44809 (local address 127.15.110.195:40199)
0927 18:47:41.202399 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.202433 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.202954 (+ 521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.203227 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.203230 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.203239 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:47:41.203737 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.203741 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.204535 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.204536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.205340 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.205342 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.205748 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.205749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.205894 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.206485 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0927 18:47:41.206485 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:47:41.206513 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0927 18:47:41.206536 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:47:41.206704 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:47:41.206710 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:47:41.206713 (+ 3us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.206772 (+ 59us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.206874 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260927 18:47:41.207549 3469 heartbeater.cc:347] Connected to a master server at 127.15.110.254:44809
I20260927 18:47:41.207653 3469 heartbeater.cc:464] Registering TS with master...
I20260927 18:47:41.207831 3469 heartbeater.cc:511] Master 127.15.110.254:44809 requested a full tablet report, sending...
I20260927 18:47:41.208175 3019 ts_manager.cc:194] Registered new tserver with Master: 814545dc07864ca68cb280a9a8cfa4a1 (127.15.110.195:40153)
I20260927 18:47:41.208546 3019 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.15.110.195:40199
I20260927 18:47:41.216015 15803 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260927 18:47:41.220279 3058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.217383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54708 (local address 127.15.110.254:44809)
0927 18:47:41.217448 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.217451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.217535 (+ 84us) server_negotiation.cc:407] Connection header received
0927 18:47:41.217607 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.217608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.217618 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.217680 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:41.217837 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.218310 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.219062 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.219304 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.219373 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.220050 (+ 677us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:41.220052 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:41.220052 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:41.220057 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:41.220070 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:41.220074 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:41.220120 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.220175 (+ 55us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.220195 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260927 18:47:41.221728 3019 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:54708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260927 18:47:41.223249 3019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260927 18:47:41.229549 3478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.225499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.194:44959 (local address 127.0.0.1:49298)
0927 18:47:41.225999 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.226009 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.226061 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.226254 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.226255 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.226257 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:41.226323 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.226325 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227593 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.227594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.228468 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.228471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.229246 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.229247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.229323 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.229337 (+ 14us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.229414 (+ 77us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.229469 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"mutex_wait_us":39,"spinlock_wait_cycles":768,"thread_start_us":96,"threads_started":1}
I20260927 18:47:41.229606 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.225763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49298 (local address 127.15.110.194:44959)
0927 18:47:41.225969 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.225975 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.226057 (+ 82us) server_negotiation.cc:407] Connection header received
0927 18:47:41.226147 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.226149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.226165 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.226256 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:41.226408 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227509 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.228561 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.229147 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.229232 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.229316 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.229483 (+ 167us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.229526 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":138,"threads_started":1}
I20260927 18:47:41.230016 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.225500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40153 (local address 127.0.0.1:36586)
0927 18:47:41.225767 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.225777 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.225832 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.226161 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.226164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.226166 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:41.226229 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.226231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227542 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.227543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.228833 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.228836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.229803 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.229804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.229860 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.229864 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.229903 (+ 39us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.229939 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":101,"thread_start_us":66,"threads_started":1}
I20260927 18:47:41.230023 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.225824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36586 (local address 127.15.110.195:40153)
0927 18:47:41.225973 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.225978 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.225988 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:41.226029 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.226031 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.226048 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.226152 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:41.226282 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227456 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.228921 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.229649 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.229794 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.229878 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.229912 (+ 34us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.229948 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260927 18:47:41.230203 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.225499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43427 (local address 127.0.0.1:33786)
0927 18:47:41.225615 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.225653 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.225739 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.226529 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.226530 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.226532 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:41.226564 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.226566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227444 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.227445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.228833 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.228836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.230000 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.230000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.230061 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.230065 (+ 4us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.230099 (+ 34us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.230140 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":76,"thread_start_us":89,"threads_started":1}
I20260927 18:47:41.230294 3272 tablet_service.cc:1467] Processing CreateTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e (DEFAULT_TABLE table=test-table [id=b523ef2dcfa4407db081ba908e947d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:41.230681 3272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f82bfd7fcf4ff0a819da19a8afe77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:41.230593 3403 tablet_service.cc:1467] Processing CreateTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e (DEFAULT_TABLE table=test-table [id=b523ef2dcfa4407db081ba908e947d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:41.230913 3403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f82bfd7fcf4ff0a819da19a8afe77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:41.230968 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.226003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33786 (local address 127.15.110.193:43427)
0927 18:47:41.226378 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.226382 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.226393 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:47:41.226429 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.226431 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.226444 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.226534 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:41.226617 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.227366 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.228941 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.229586 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.230708 (+ 1122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.230821 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.230847 (+ 26us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.230891 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":89,"threads_started":1}
I20260927 18:47:41.231506 3141 tablet_service.cc:1467] Processing CreateTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e (DEFAULT_TABLE table=test-table [id=b523ef2dcfa4407db081ba908e947d59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260927 18:47:41.231812 3141 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14f82bfd7fcf4ff0a819da19a8afe77e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:41.234169 3484 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Bootstrap starting.
I20260927 18:47:41.234534 3485 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: Bootstrap starting.
I20260927 18:47:41.234973 3484 tablet_bootstrap.cc:654] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:41.235172 3485 tablet_bootstrap.cc:654] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:41.235313 3483 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Bootstrap starting.
I20260927 18:47:41.235402 3484 log.cc:824] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:41.235543 3485 log.cc:824] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:41.235981 3483 tablet_bootstrap.cc:654] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:41.236248 3485 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: No bootstrap required, opened a new log
I20260927 18:47:41.236248 3484 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: No bootstrap required, opened a new log
I20260927 18:47:41.236258 3483 log.cc:824] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:41.236320 3484 ts_tablet_manager.cc:1397] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:47:41.236320 3485 ts_tablet_manager.cc:1397] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260927 18:47:41.236943 3483 tablet_bootstrap.cc:492] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: No bootstrap required, opened a new log
I20260927 18:47:41.237013 3483 ts_tablet_manager.cc:1397] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260927 18:47:41.238914 3485 raft_consensus.cc:348] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.238914 3484 raft_consensus.cc:348] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.239089 3485 raft_consensus.cc:374] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:41.239100 3484 raft_consensus.cc:374] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:41.239140 3484 raft_consensus.cc:729] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ac463f217284abf944d9c46dd081c70, State: Initialized, Role: FOLLOWER
I20260927 18:47:41.239140 3485 raft_consensus.cc:729] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c5eba051654ca99b0ccb7934d0a635, State: Initialized, Role: FOLLOWER
I20260927 18:47:41.239125 3483 raft_consensus.cc:348] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.239295 3483 raft_consensus.cc:374] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:41.239338 3483 raft_consensus.cc:729] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 814545dc07864ca68cb280a9a8cfa4a1, State: Initialized, Role: FOLLOWER
I20260927 18:47:41.239310 3484 consensus_queue.cc:260] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.239372 3485 consensus_queue.cc:260] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.239475 3483 consensus_queue.cc:260] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.239571 3484 ts_tablet_manager.cc:1428] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:47:41.239593 3485 ts_tablet_manager.cc:1428] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260927 18:47:41.239696 3207 heartbeater.cc:503] Master 127.15.110.254:44809 was elected leader, sending a full tablet report...
I20260927 18:47:41.239696 3338 heartbeater.cc:503] Master 127.15.110.254:44809 was elected leader, sending a full tablet report...
I20260927 18:47:41.239732 3483 ts_tablet_manager.cc:1428] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260927 18:47:41.239852 3469 heartbeater.cc:503] Master 127.15.110.254:44809 was elected leader, sending a full tablet report...
W20260927 18:47:41.320923 3339 tablet.cc:2369] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260927 18:47:41.332464 3489 raft_consensus.cc:482] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260927 18:47:41.332592 3489 raft_consensus.cc:504] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.332926 3489 leader_election.cc:288] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 39c5eba051654ca99b0ccb7934d0a635 (127.15.110.193:43427), 814545dc07864ca68cb280a9a8cfa4a1 (127.15.110.195:40153)
I20260927 18:47:41.337037 3482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.333236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:47863 (local address 127.15.110.193:43427)
0927 18:47:41.333334 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.333336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.333348 (+ 12us) server_negotiation.cc:407] Connection header received
0927 18:47:41.333387 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.333388 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.333398 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.333453 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:41.333647 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.334517 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336049 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.336662 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336782 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.336790 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.336920 (+ 130us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.336961 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260927 18:47:41.337037 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.333253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.15.110.194:55303 (local address 127.15.110.195:40153)
0927 18:47:41.333317 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.333319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.333488 (+ 169us) server_negotiation.cc:407] Connection header received
0927 18:47:41.333529 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.333530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.333537 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.333583 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0927 18:47:41.333741 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.334516 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336055 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.336662 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336748 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.336760 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.336920 (+ 160us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.336961 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260927 18:47:41.337054 3340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.333135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43427 (local address 127.15.110.194:47863)
0927 18:47:41.333207 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.333222 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.333290 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.333475 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.333476 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.333477 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:41.333554 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.333556 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.334612 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.334613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.335956 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.335959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.336793 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.336853 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.336855 (+ 2us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.336905 (+ 50us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.336949 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":46}
I20260927 18:47:41.337064 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.333138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.195:40153 (local address 127.15.110.194:55303)
0927 18:47:41.333394 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.333403 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:47:41.333474 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:47:41.333598 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:47:41.333598 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:47:41.333600 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0927 18:47:41.333673 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.333674 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.334611 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.334613 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.335956 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:47:41.335959 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.336741 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.336742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:47:41.336831 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.336836 (+ 5us) client_negotiation.cc:769] Sending connection context
0927 18:47:41.336903 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:47:41.336948 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":118,"thread_start_us":108,"threads_started":1}
I20260927 18:47:41.337491 3161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f82bfd7fcf4ff0a819da19a8afe77e" candidate_uuid: "8ac463f217284abf944d9c46dd081c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c5eba051654ca99b0ccb7934d0a635" is_pre_election: true
I20260927 18:47:41.337675 3161 raft_consensus.cc:2454] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac463f217284abf944d9c46dd081c70 in term 0.
I20260927 18:47:41.337677 3423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f82bfd7fcf4ff0a819da19a8afe77e" candidate_uuid: "8ac463f217284abf944d9c46dd081c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814545dc07864ca68cb280a9a8cfa4a1" is_pre_election: true
I20260927 18:47:41.337829 3423 raft_consensus.cc:2454] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8ac463f217284abf944d9c46dd081c70 in term 0.
I20260927 18:47:41.337939 3227 leader_election.cc:302] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 39c5eba051654ca99b0ccb7934d0a635, 8ac463f217284abf944d9c46dd081c70; no voters:
I20260927 18:47:41.338129 3489 raft_consensus.cc:2781] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260927 18:47:41.338198 3489 raft_consensus.cc:482] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260927 18:47:41.338233 3489 raft_consensus.cc:3037] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:41.338958 3489 raft_consensus.cc:504] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.339103 3489 leader_election.cc:288] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [CANDIDATE]: Term 1 election: Requested vote from peers 39c5eba051654ca99b0ccb7934d0a635 (127.15.110.193:43427), 814545dc07864ca68cb280a9a8cfa4a1 (127.15.110.195:40153)
I20260927 18:47:41.339265 3161 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f82bfd7fcf4ff0a819da19a8afe77e" candidate_uuid: "8ac463f217284abf944d9c46dd081c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "39c5eba051654ca99b0ccb7934d0a635"
I20260927 18:47:41.339327 3423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14f82bfd7fcf4ff0a819da19a8afe77e" candidate_uuid: "8ac463f217284abf944d9c46dd081c70" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "814545dc07864ca68cb280a9a8cfa4a1"
I20260927 18:47:41.339365 3161 raft_consensus.cc:3037] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:41.339423 3423 raft_consensus.cc:3037] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:41.340091 3423 raft_consensus.cc:2454] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac463f217284abf944d9c46dd081c70 in term 1.
I20260927 18:47:41.340160 3161 raft_consensus.cc:2454] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8ac463f217284abf944d9c46dd081c70 in term 1.
I20260927 18:47:41.340298 3225 leader_election.cc:302] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 814545dc07864ca68cb280a9a8cfa4a1, 8ac463f217284abf944d9c46dd081c70; no voters:
I20260927 18:47:41.340387 3489 raft_consensus.cc:2781] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:41.340521 3489 raft_consensus.cc:686] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 1 LEADER]: Becoming Leader. State: Replica: 8ac463f217284abf944d9c46dd081c70, State: Running, Role: LEADER
I20260927 18:47:41.340621 3489 consensus_queue.cc:237] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } }
I20260927 18:47:41.341426 3017 catalog_manager.cc:5495] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8ac463f217284abf944d9c46dd081c70 (127.15.110.194). New cstate: current_term: 1 leader_uuid: "8ac463f217284abf944d9c46dd081c70" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ac463f217284abf944d9c46dd081c70" member_type: VOTER last_known_addr { host: "127.15.110.194" port: 44959 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 } health_report { overall_health: UNKNOWN } } }
I20260927 18:47:41.349906 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.347751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.15.110.194:44959)
0927 18:47:41.347806 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:41.347808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:41.347818 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:47:41.347853 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:41.347853 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:41.347870 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:41.347945 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:47:41.348027 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.348517 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.349285 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:41.349480 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:41.349538 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:41.349651 (+ 113us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:47:41.349741 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:47:41.349794 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:41.349824 (+ 30us) server_negotiation.cc:299] Negotiation successful
0927 18:47:41.349838 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:47:41.351643 3161 raft_consensus.cc:1261] T 14f82bfd7fcf4ff0a819da19a8afe77e P 39c5eba051654ca99b0ccb7934d0a635 [term 1 FOLLOWER]: Refusing update from remote peer 8ac463f217284abf944d9c46dd081c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:41.351643 3423 raft_consensus.cc:1261] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 1 FOLLOWER]: Refusing update from remote peer 8ac463f217284abf944d9c46dd081c70: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:47:41.351970 3493 consensus_queue.cc:1035] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "39c5eba051654ca99b0ccb7934d0a635" member_type: VOTER last_known_addr { host: "127.15.110.193" port: 43427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:41.352097 3489 consensus_queue.cc:1035] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [LEADER]: Connected to new peer: Peer: permanent_uuid: "814545dc07864ca68cb280a9a8cfa4a1" member_type: VOTER last_known_addr { host: "127.15.110.195" port: 40153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:47:41.354606 3497 mvcc.cc:205] Tried to move back new op lower bound from 7334030792093736960 to 7334030792051773440. Current Snapshot: MvccSnapshot[applied={T|T < 7334030792093736960}]
I20260927 18:47:41.354890 3496 mvcc.cc:205] Tried to move back new op lower bound from 7334030792093736960 to 7334030792051773440. Current Snapshot: MvccSnapshot[applied={T|T < 7334030792093736960 or (T in {7334030792093736960})}]
I20260927 18:47:41.355355 3498 mvcc.cc:205] Tried to move back new op lower bound from 7334030792093736960 to 7334030792051773440. Current Snapshot: MvccSnapshot[applied={T|T < 7334030792093736960}]
I20260927 18:47:41.356598 3017 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:54708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:41.356695 3017 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:54708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260927 18:47:41.357371 3017 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7e7d24647664fc5806671c43b8a4bdc: Sending DeleteTablet for 3 replicas of tablet 14f82bfd7fcf4ff0a819da19a8afe77e
I20260927 18:47:41.357722 3403 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:41 UTC) from {username='slave'} at 127.0.0.1:36586
I20260927 18:47:41.357736 3272 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:41 UTC) from {username='slave'} at 127.0.0.1:49298
I20260927 18:47:41.357748 3141 tablet_service.cc:1514] Processing DeleteTablet for tablet 14f82bfd7fcf4ff0a819da19a8afe77e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-27 18:47:41 UTC) from {username='slave'} at 127.0.0.1:33786
I20260927 18:47:41.357934 3506 tablet_replica.cc:330] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: stopping tablet replica
I20260927 18:47:41.358026 3506 raft_consensus.cc:2229] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260927 18:47:41.358174 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3079
I20260927 18:47:41.358868 3507 tablet_replica.cc:330] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: stopping tablet replica
I20260927 18:47:41.358956 3507 raft_consensus.cc:2229] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 1 LEADER]: Raft consensus shutting down.
I20260927 18:47:41.359234 3507 raft_consensus.cc:2258] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:41.359660 3507 ts_tablet_manager.cc:1905] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:41.358235 3506 raft_consensus.cc:2258] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260927 18:47:41.361212 3507 ts_tablet_manager.cc:1918] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:41.361330 3507 log.cc:1196] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-1/wal/wals/14f82bfd7fcf4ff0a819da19a8afe77e
I20260927 18:47:41.361569 3506 ts_tablet_manager.cc:1905] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260927 18:47:41.361703 3507 ts_tablet_manager.cc:1939] T 14f82bfd7fcf4ff0a819da19a8afe77e P 8ac463f217284abf944d9c46dd081c70: Deleting consensus metadata
I20260927 18:47:41.362120 3006 catalog_manager.cc:4841] TS 8ac463f217284abf944d9c46dd081c70 (127.15.110.194:44959): tablet 14f82bfd7fcf4ff0a819da19a8afe77e (table test-table [id=b523ef2dcfa4407db081ba908e947d59]) successfully deleted
I20260927 18:47:41.363260 3506 ts_tablet_manager.cc:1918] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260927 18:47:41.363344 3506 log.cc:1196] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Deleting WAL directory at /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790534620099873-15803-0/minicluster-data/ts-2/wal/wals/14f82bfd7fcf4ff0a819da19a8afe77e
I20260927 18:47:41.363620 3506 ts_tablet_manager.cc:1939] T 14f82bfd7fcf4ff0a819da19a8afe77e P 814545dc07864ca68cb280a9a8cfa4a1: Deleting consensus metadata
I20260927 18:47:41.364037 3003 catalog_manager.cc:4841] TS 814545dc07864ca68cb280a9a8cfa4a1 (127.15.110.195:40153): tablet 14f82bfd7fcf4ff0a819da19a8afe77e (table test-table [id=b523ef2dcfa4407db081ba908e947d59]) successfully deleted
W20260927 18:47:41.364334 3005 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.15.110.193:43427 (error 108)
I20260927 18:47:41.364656 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3210
I20260927 18:47:41.365521 3477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:41.365382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.193:43427 (local address 127.0.0.1:33800)
0927 18:47:41.365442 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:47:41.365487 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.193:43427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:47:41.365635 3005 catalog_manager.cc:4584] TS 39c5eba051654ca99b0ccb7934d0a635 (127.15.110.193:43427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14f82bfd7fcf4ff0a819da19a8afe77e: Network error: Client connection negotiation failed: client connection to 127.15.110.193:43427: connect: Connection refused (error 111)
I20260927 18:47:41.370299 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3341
I20260927 18:47:41.374791 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 2988
2026-09-27T18:47:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (759 ms)
[----------] 6 tests from EncryptionPolicyTest (8644 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:41 dist-test-slave-vs05 krb5kdc[3512](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:41 dist-test-slave-vs05 krb5kdc[3512](info): set up 2 sockets
Sep 27 18:47:41 dist-test-slave-vs05 krb5kdc[3512](info): commencing operation
krb5kdc: starting...
W20260927 18:47:42.419365 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:42.541687 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37959
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37959
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:42.622375 3525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:42.622566 3525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:42.622588 3525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:42.623714 3525 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:42.623754 3525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:42.623767 3525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:42.623780 3525 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:42.623798 3525 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:42.625159 3525 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37959
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37959
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:42.625335 3525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:42.625504 3525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:42.630277 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:42.630277 3531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:42.630316 3530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:42.630293 3525 server_base.cc:1034] running on GCE node
I20260927 18:47:42.630707 3525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:42.630991 3525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:42.632148 3525 hybrid_clock.cc:648] HybridClock initialized: now 1790534862632127 us; error 50 us; skew 500 ppm
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3512](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534862, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:42.644897 3525 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:42.645342 3525 webserver.cc:460] Webserver started at http://127.0.0.1:42959/ using document root <none> and password file <none>
I20260927 18:47:42.645579 3525 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:42.645645 3525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:42.645793 3525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:42.646922 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "98e751ecd79c475390de6b3b77f270fd"
format_stamp: "Formatted at 2026-09-27 18:47:42 on dist-test-slave-vs05"
server_key: "7a21e7a55045799ba6903d179946e3a1"
server_key_iv: "e63350e9c50938da78d35f4a9739a644"
server_key_version: "encryptionkey@0"
I20260927 18:47:42.647302 3525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "98e751ecd79c475390de6b3b77f270fd"
format_stamp: "Formatted at 2026-09-27 18:47:42 on dist-test-slave-vs05"
server_key: "7a21e7a55045799ba6903d179946e3a1"
server_key_iv: "e63350e9c50938da78d35f4a9739a644"
server_key_version: "encryptionkey@0"
I20260927 18:47:42.648643 3525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:42.649386 3540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:42.649552 3525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:42.649648 3525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "98e751ecd79c475390de6b3b77f270fd"
format_stamp: "Formatted at 2026-09-27 18:47:42 on dist-test-slave-vs05"
server_key: "7a21e7a55045799ba6903d179946e3a1"
server_key_iv: "e63350e9c50938da78d35f4a9739a644"
server_key_version: "encryptionkey@0"
I20260927 18:47:42.649724 3525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:42.669831 3525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:42.671890 3525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:42.672050 3525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:42.676038 3525 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37959
I20260927 18:47:42.676072 3592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37959 every 8 connection(s)
I20260927 18:47:42.676450 3525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:42.676720 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3525
I20260927 18:47:42.676847 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:42.677026 15803 external_mini_cluster.cc:1428] Setting key 500bcd8f7a6f53b18cba173db36cc98b
I20260927 18:47:42.677484 3593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:42.680238 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd: Bootstrap starting.
I20260927 18:47:42.680824 3593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:42.681113 3593 log.cc:824] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:42.681743 3593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd: No bootstrap required, opened a new log
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3512](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:47:42.683630 3593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } }
I20260927 18:47:42.683781 3593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:42.683840 3593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 98e751ecd79c475390de6b3b77f270fd, State: Initialized, Role: FOLLOWER
I20260927 18:47:42.683997 3593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } }
I20260927 18:47:42.684084 3593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:42.684128 3593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:42.684177 3593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:42.684702 3593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } }
I20260927 18:47:42.684813 3593 leader_election.cc:302] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 98e751ecd79c475390de6b3b77f270fd; no voters:
I20260927 18:47:42.684976 3593 leader_election.cc:288] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:42.685029 3598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:42.685171 3598 raft_consensus.cc:686] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [term 1 LEADER]: Becoming Leader. State: Replica: 98e751ecd79c475390de6b3b77f270fd, State: Running, Role: LEADER
I20260927 18:47:42.685240 3593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:42.685256 3598 consensus_queue.cc:237] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } }
I20260927 18:47:42.685555 3599 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "98e751ecd79c475390de6b3b77f270fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } } }
I20260927 18:47:42.685645 3599 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:42.685806 3600 sys_catalog.cc:455] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 98e751ecd79c475390de6b3b77f270fd. Latest consensus state: current_term: 1 leader_uuid: "98e751ecd79c475390de6b3b77f270fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "98e751ecd79c475390de6b3b77f270fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37959 } } }
I20260927 18:47:42.685878 3600 sys_catalog.cc:458] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:42.685986 3604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:42.686532 3604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:42.687914 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:42.677578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46233 (local address 127.0.0.1:37959)
0927 18:47:42.677765 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:42.677774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:42.677799 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:47:42.678227 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:42.678233 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:42.678278 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:42.678429 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:42.679087 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:42.680014 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:42.680818 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:42.681320 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:42.681436 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:42.683790 (+ 2354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:42.683794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:42.683794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:42.683806 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:42.685851 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:42.686429 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:42.686432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:42.686432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:42.686533 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:42.686836 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:42.686837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:42.686837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:42.687150 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:42.687332 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:42.687615 (+ 283us) server_negotiation.cc:299] Negotiation successful
0927 18:47:42.687725 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":89,"threads_started":1}
I20260927 18:47:42.688342 3604 catalog_manager.cc:1347] Generated new cluster ID: 388ace503630441e8396cd38a559f4e0
I20260927 18:47:42.688393 3604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:42.688817 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:42.697413 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:42.690136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40195 (local address 127.0.0.1:37959)
0927 18:47:42.690177 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:42.690179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:42.690343 (+ 164us) server_negotiation.cc:407] Connection header received
0927 18:47:42.690396 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:42.690397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:42.690407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:42.690473 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:42.691037 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:42.691863 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:42.692843 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:42.693175 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:42.693250 (+ 75us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:42.694588 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:42.694590 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:42.694590 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:42.694597 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:42.696230 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:42.696627 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:42.696628 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:42.696628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:42.696687 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:42.696900 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:42.696901 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:42.696902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:42.697033 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:42.697138 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:42.697250 (+ 112us) server_negotiation.cc:299] Negotiation successful
0927 18:47:42.697312 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:47:42.698837 3604 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:42.699375 3604 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:42.701479 3604 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 98e751ecd79c475390de6b3b77f270fd: Generated new TSK 0
I20260927 18:47:42.701681 3604 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:42.707942 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3525
W20260927 18:47:42.712097 3510 connection.cc:535] client connection to 127.0.0.1:37959 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37959): Connection reset by peer (error 104)
2026-09-27T18:47:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3624](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3624](info): set up 2 sockets
Sep 27 18:47:42 dist-test-slave-vs05 krb5kdc[3624](info): commencing operation
krb5kdc: starting...
W20260927 18:47:43.811261 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:43 dist-test-slave-vs05 krb5kdc[3624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:43.930526 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:44.011314 3637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:44.011516 3637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:44.011540 3637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:44.012832 3637 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:44.012897 3637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:44.012923 3637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:44.012969 3637 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:44.013001 3637 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:44.014777 3637 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:44.015053 3637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:44.015293 3637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:44.020022 3643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:44.020041 3645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:44.020051 3642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:44.020361 3637 server_base.cc:1034] running on GCE node
I20260927 18:47:44.020610 3637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:44.020885 3637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:44.022042 3637 hybrid_clock.cc:648] HybridClock initialized: now 1790534864022024 us; error 25 us; skew 500 ppm
Sep 27 18:47:44 dist-test-slave-vs05 krb5kdc[3624](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534864, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:44.024554 3637 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:44.025048 3637 webserver.cc:460] Webserver started at http://127.0.0.1:33501/ using document root <none> and password file <none>
I20260927 18:47:44.025280 3637 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:44.025344 3637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:44.025493 3637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:44.026602 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "ea24d88c351c48d595578de687d08a90"
format_stamp: "Formatted at 2026-09-27 18:47:44 on dist-test-slave-vs05"
server_key: "74d8a18bdcee10aa8a8f4888f3fe00ba"
server_key_iv: "ee087dc92377423b4fa3207d1d9a05c4"
server_key_version: "encryptionkey@0"
I20260927 18:47:44.026999 3637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "ea24d88c351c48d595578de687d08a90"
format_stamp: "Formatted at 2026-09-27 18:47:44 on dist-test-slave-vs05"
server_key: "74d8a18bdcee10aa8a8f4888f3fe00ba"
server_key_iv: "ee087dc92377423b4fa3207d1d9a05c4"
server_key_version: "encryptionkey@0"
I20260927 18:47:44.028389 3637 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:44.029181 3652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:44.029353 3637 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:47:44.029453 3637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "ea24d88c351c48d595578de687d08a90"
format_stamp: "Formatted at 2026-09-27 18:47:44 on dist-test-slave-vs05"
server_key: "74d8a18bdcee10aa8a8f4888f3fe00ba"
server_key_iv: "ee087dc92377423b4fa3207d1d9a05c4"
server_key_version: "encryptionkey@0"
I20260927 18:47:44.029522 3637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:44.056247 3637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:44.058251 3637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:44.058416 3637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:44.062312 3637 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39703
I20260927 18:47:44.062371 3704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39703 every 8 connection(s)
I20260927 18:47:44.062731 3637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:44.063745 3705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:44.065534 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3637
I20260927 18:47:44.065667 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:44.065861 15803 external_mini_cluster.cc:1428] Setting key 5ef28ba1f6c43a80a0a562a2d9d42a90
I20260927 18:47:44.066568 3705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90: Bootstrap starting.
I20260927 18:47:44.067231 3705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:44.067593 3705 log.cc:824] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:44.068266 3705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90: No bootstrap required, opened a new log
I20260927 18:47:44.070647 3705 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260927 18:47:44.070827 3705 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:44.070866 3705 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea24d88c351c48d595578de687d08a90, State: Initialized, Role: FOLLOWER
I20260927 18:47:44.071014 3705 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260927 18:47:44.071082 3705 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:44.071122 3705 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:44.071194 3705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:44.071609 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:44.066391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54639 (local address 127.0.0.1:39703)
0927 18:47:44.066560 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:44.066568 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:44.066593 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:47:44.067036 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:44.067043 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:44.067091 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:44.067266 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:44.067488 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:44.068443 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:44.069524 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:44.069973 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:44.070061 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:44.070802 (+ 741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:44.070805 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:44.070806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:44.071071 (+ 265us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:44.071123 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:44.071137 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:44.071261 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:44.071357 (+ 96us) server_negotiation.cc:299] Negotiation successful
0927 18:47:44.071398 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":85,"threads_started":1}
I20260927 18:47:44.071841 3705 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260927 18:47:44.071961 3705 leader_election.cc:302] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea24d88c351c48d595578de687d08a90; no voters:
I20260927 18:47:44.072115 3705 leader_election.cc:288] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:44.072142 3710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:44.072299 3710 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [term 1 LEADER]: Becoming Leader. State: Replica: ea24d88c351c48d595578de687d08a90, State: Running, Role: LEADER
I20260927 18:47:44.072381 3710 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260927 18:47:44.072402 3705 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:44.072620 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:44.072791 3710 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea24d88c351c48d595578de687d08a90. Latest consensus state: current_term: 1 leader_uuid: "ea24d88c351c48d595578de687d08a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } } }
I20260927 18:47:44.072888 3710 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:44.072791 3711 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea24d88c351c48d595578de687d08a90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea24d88c351c48d595578de687d08a90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } } }
I20260927 18:47:44.073096 3711 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:44.073225 3717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:44.073889 3717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:44.075469 3717 catalog_manager.cc:1347] Generated new cluster ID: 1ba05ee504d846b1bbef95bbeb5df085
I20260927 18:47:44.075521 3717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:44.077752 3708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:44.074209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41441 (local address 127.0.0.1:39703)
0927 18:47:44.074246 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:44.074248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:44.074503 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:47:44.074899 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:44.074901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:44.074913 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:44.074976 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:44.075165 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:44.075849 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:44.076580 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:44.076798 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:44.076878 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:44.077515 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:44.077517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:44.077517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:44.077522 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:44.077535 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:44.077539 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:44.077592 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:44.077651 (+ 59us) server_negotiation.cc:299] Negotiation successful
0927 18:47:44.077660 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:47:44.099833 3717 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:44.100482 3717 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:44.109218 3717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea24d88c351c48d595578de687d08a90: Generated new TSK 0
I20260927 18:47:44.109417 3717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:44.115532 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3637
W20260927 18:47:44.120062 3622 connection.cc:535] client connection to 127.0.0.1:39703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39703): Connection reset by peer (error 104)
2026-09-27T18:47:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:44 dist-test-slave-vs05 krb5kdc[3736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:44 dist-test-slave-vs05 krb5kdc[3736](info): set up 2 sockets
Sep 27 18:47:44 dist-test-slave-vs05 krb5kdc[3736](info): commencing operation
krb5kdc: starting...
W20260927 18:47:45.151340 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:45 dist-test-slave-vs05 krb5kdc[3736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534865, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:45.265978 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:45.345070 3749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:45.345276 3749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:45.345299 3749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:45.346441 3749 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:45.346483 3749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:45.346498 3749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:45.346510 3749 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:45.346522 3749 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:45.348011 3749 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42509
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3749
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:45.348253 3749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:45.348482 3749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:45.353190 3755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:45.353183 3754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:45.353266 3749 server_base.cc:1034] running on GCE node
W20260927 18:47:45.353186 3757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:45.353605 3749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:45.353894 3749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:45.355063 3749 hybrid_clock.cc:648] HybridClock initialized: now 1790534865355045 us; error 26 us; skew 500 ppm
Sep 27 18:47:45 dist-test-slave-vs05 krb5kdc[3736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534865, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:45.357650 3749 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:45.358088 3749 webserver.cc:460] Webserver started at http://127.0.0.1:45055/ using document root <none> and password file <none>
I20260927 18:47:45.358327 3749 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:45.358394 3749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:45.358556 3749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:45.359668 3749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "d51eebaa92d547fe8575b17e428533f3"
format_stamp: "Formatted at 2026-09-27 18:47:45 on dist-test-slave-vs05"
server_key: "bca4c7f2035a8068454e0e730be4aadd"
server_key_iv: "7ae4d80f938cff6b692e2b9bf8f22b7a"
server_key_version: "encryptionkey@0"
I20260927 18:47:45.360077 3749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "d51eebaa92d547fe8575b17e428533f3"
format_stamp: "Formatted at 2026-09-27 18:47:45 on dist-test-slave-vs05"
server_key: "bca4c7f2035a8068454e0e730be4aadd"
server_key_iv: "7ae4d80f938cff6b692e2b9bf8f22b7a"
server_key_version: "encryptionkey@0"
I20260927 18:47:45.361359 3749 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:45.362087 3764 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:45.362242 3749 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:45.362349 3749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "d51eebaa92d547fe8575b17e428533f3"
format_stamp: "Formatted at 2026-09-27 18:47:45 on dist-test-slave-vs05"
server_key: "bca4c7f2035a8068454e0e730be4aadd"
server_key_iv: "7ae4d80f938cff6b692e2b9bf8f22b7a"
server_key_version: "encryptionkey@0"
I20260927 18:47:45.362414 3749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:45.365716 3749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:45.367978 3749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:45.368125 3749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:45.371944 3749 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42509
I20260927 18:47:45.371987 3816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42509 every 8 connection(s)
I20260927 18:47:45.372323 3749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:45.373181 3817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:45.375417 3817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: Bootstrap starting.
I20260927 18:47:45.375962 3817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:45.376223 3817 log.cc:824] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:45.376878 3817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: No bootstrap required, opened a new log
I20260927 18:47:45.378405 3817 raft_consensus.cc:348] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260927 18:47:45.378536 3817 raft_consensus.cc:374] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:45.378558 3817 raft_consensus.cc:729] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d51eebaa92d547fe8575b17e428533f3, State: Initialized, Role: FOLLOWER
I20260927 18:47:45.378718 3817 consensus_queue.cc:260] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260927 18:47:45.378777 3817 raft_consensus.cc:388] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:45.378798 3817 raft_consensus.cc:482] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:45.378842 3817 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:45.379427 3817 raft_consensus.cc:504] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260927 18:47:45.379531 3817 leader_election.cc:302] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d51eebaa92d547fe8575b17e428533f3; no voters:
I20260927 18:47:45.379701 3817 leader_election.cc:288] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:45.379755 3820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:45.379920 3820 raft_consensus.cc:686] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [term 1 LEADER]: Becoming Leader. State: Replica: d51eebaa92d547fe8575b17e428533f3, State: Running, Role: LEADER
I20260927 18:47:45.379976 3817 sys_catalog.cc:564] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:45.380200 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3749
I20260927 18:47:45.380319 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:45.380491 15803 external_mini_cluster.cc:1428] Setting key 968eedd82970aa426f64245921ce80f7
I20260927 18:47:45.380008 3820 consensus_queue.cc:237] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } }
I20260927 18:47:45.381255 3821 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d51eebaa92d547fe8575b17e428533f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260927 18:47:45.381289 3822 sys_catalog.cc:455] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d51eebaa92d547fe8575b17e428533f3. Latest consensus state: current_term: 1 leader_uuid: "d51eebaa92d547fe8575b17e428533f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d51eebaa92d547fe8575b17e428533f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42509 } } }
I20260927 18:47:45.381351 3821 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:45.381398 3822 sys_catalog.cc:458] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3 [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:45.382290 3836 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:45.382345 3836 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:45.382442 3837 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:45.382623 3837 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:45.383170 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:45.381029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53515 (local address 127.0.0.1:42509)
0927 18:47:45.381567 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:45.381578 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:45.381703 (+ 125us) server_negotiation.cc:407] Connection header received
0927 18:47:45.381849 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:45.381855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:45.381886 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:45.382001 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:45.382748 (+ 747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:45.382752 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:45.382752 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:45.382758 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:45.382795 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:45.382808 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:45.382884 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:45.382956 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:47:45.382993 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":65,"threads_started":1}
I20260927 18:47:45.384047 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:45.384284 3837 catalog_manager.cc:1347] Generated new cluster ID: 3d2174109c12420ea18fef972dd92961
I20260927 18:47:45.384321 3837 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:45.393695 3837 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:45.393874 3837 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:45.396029 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:45.387560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54433 (local address 127.0.0.1:42509)
0927 18:47:45.387612 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:45.387615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:45.393402 (+ 5787us) server_negotiation.cc:407] Connection header received
0927 18:47:45.393483 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:45.393484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:45.393501 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:45.393605 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:45.395792 (+ 2187us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:45.395794 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:45.395794 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:45.395799 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:45.395817 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:45.395821 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:45.395883 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:45.395944 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 18:47:45.395958 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260927 18:47:45.396329 3837 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d51eebaa92d547fe8575b17e428533f3: Generated new TSK 0
I20260927 18:47:45.396486 3837 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:45.401263 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3749
2026-09-27T18:47:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1284 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:45 dist-test-slave-vs05 krb5kdc[3848](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:45 dist-test-slave-vs05 krb5kdc[3848](info): set up 2 sockets
Sep 27 18:47:45 dist-test-slave-vs05 krb5kdc[3848](info): commencing operation
krb5kdc: starting...
W20260927 18:47:46.431380 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:46.550019 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35293
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:46.630317 3861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:46.630550 3861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:46.630591 3861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:46.631767 3861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:46.631831 3861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:46.631866 3861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:46.631904 3861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:46.631934 3861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:46.633407 3861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:46.633672 3861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:46.633900 3861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:47:46.639279 3861 server_base.cc:1034] running on GCE node
W20260927 18:47:46.639212 3869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:46.639249 3866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:46.639228 3867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:46.639658 3861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:46.639971 3861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:46.641134 3861 hybrid_clock.cc:648] HybridClock initialized: now 1790534866641120 us; error 41 us; skew 500 ppm
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3848](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534866, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:46.643581 3861 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:46.644055 3861 webserver.cc:460] Webserver started at http://127.0.0.1:44655/ using document root <none> and password file <none>
I20260927 18:47:46.644280 3861 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:46.644345 3861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:46.644495 3861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:46.645563 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "dd0bafd3d44546b5ba805a5d89c3d889"
format_stamp: "Formatted at 2026-09-27 18:47:46 on dist-test-slave-vs05"
server_key: "574e2cfb2fed3f5f67f8f92e674a4913"
server_key_iv: "e3f5b8a88807014b4c59433a1bd0fa37"
server_key_version: "encryptionkey@0"
I20260927 18:47:46.645927 3861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "dd0bafd3d44546b5ba805a5d89c3d889"
format_stamp: "Formatted at 2026-09-27 18:47:46 on dist-test-slave-vs05"
server_key: "574e2cfb2fed3f5f67f8f92e674a4913"
server_key_iv: "e3f5b8a88807014b4c59433a1bd0fa37"
server_key_version: "encryptionkey@0"
I20260927 18:47:46.647310 3861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:46.648063 3876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:46.648233 3861 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:46.648330 3861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "dd0bafd3d44546b5ba805a5d89c3d889"
format_stamp: "Formatted at 2026-09-27 18:47:46 on dist-test-slave-vs05"
server_key: "574e2cfb2fed3f5f67f8f92e674a4913"
server_key_iv: "e3f5b8a88807014b4c59433a1bd0fa37"
server_key_version: "encryptionkey@0"
I20260927 18:47:46.648415 3861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:46.661124 3861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:46.662911 3861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:46.663074 3861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:46.667003 3861 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35293
I20260927 18:47:46.667035 3928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35293 every 8 connection(s)
I20260927 18:47:46.667476 3861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:46.668287 3929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:46.670478 3929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: Bootstrap starting.
I20260927 18:47:46.670984 3929 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:46.671289 3929 log.cc:824] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:46.671892 3929 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: No bootstrap required, opened a new log
I20260927 18:47:46.673386 3929 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260927 18:47:46.673523 3929 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:46.673545 3929 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd0bafd3d44546b5ba805a5d89c3d889, State: Initialized, Role: FOLLOWER
I20260927 18:47:46.673678 3929 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260927 18:47:46.673759 3929 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:46.673784 3929 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:46.673813 3929 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:46.674290 3929 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260927 18:47:46.674427 3929 leader_election.cc:302] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd0bafd3d44546b5ba805a5d89c3d889; no voters:
I20260927 18:47:46.674544 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3861
I20260927 18:47:46.674615 3929 leader_election.cc:288] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:46.674669 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:46.674656 3932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:46.674813 3932 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [term 1 LEADER]: Becoming Leader. State: Replica: dd0bafd3d44546b5ba805a5d89c3d889, State: Running, Role: LEADER
I20260927 18:47:46.674844 15803 external_mini_cluster.cc:1428] Setting key 7d6406d105c715754dd2d3044d606339
I20260927 18:47:46.674865 3929 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:46.674909 3932 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } }
I20260927 18:47:46.675580 3934 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd0bafd3d44546b5ba805a5d89c3d889. Latest consensus state: current_term: 1 leader_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } } }
I20260927 18:47:46.675629 3933 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd0bafd3d44546b5ba805a5d89c3d889" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35293 } } }
I20260927 18:47:46.675976 3933 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:46.675972 3934 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889 [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:46.676717 3950 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:46.676774 3950 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:46.676874 3948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:46.677086 3948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:46.678704 3948 catalog_manager.cc:1347] Generated new cluster ID: d5204a9c2b1f496c8dc29cdfafdfee32
I20260927 18:47:46.678761 3948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3848](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534866, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:47:46.686761 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:46.675362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42465 (local address 127.0.0.1:35293)
0927 18:47:46.676135 (+ 773us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:46.676145 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:46.676180 (+ 35us) server_negotiation.cc:407] Connection header received
0927 18:47:46.676308 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:46.676315 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:46.676377 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:46.676525 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:46.677298 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:46.678507 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:46.679629 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:46.680261 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:46.680380 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:46.683162 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:46.683166 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:46.683166 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:46.683198 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:46.685143 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:46.685550 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:46.685553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:46.685553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:46.685663 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:46.685868 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:46.685868 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:46.685869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:46.686154 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:46.686303 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:46.686410 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 18:47:46.686541 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":724,"thread_start_us":78,"threads_started":1}
I20260927 18:47:46.687965 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:46.688413 3948 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:46.688897 3948 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:46.692492 3948 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd0bafd3d44546b5ba805a5d89c3d889: Generated new TSK 0
I20260927 18:47:46.692664 3948 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:46.696346 3940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:46.689573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58787 (local address 127.0.0.1:35293)
0927 18:47:46.689605 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:46.689607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:46.689701 (+ 94us) server_negotiation.cc:407] Connection header received
0927 18:47:46.689783 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:46.689784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:46.689792 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:46.689819 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:46.690515 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:46.691275 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:46.692341 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:46.692647 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:46.692713 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:46.693820 (+ 1107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:46.693821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:46.693821 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:46.693828 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:46.695326 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:46.695632 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:46.695633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:46.695634 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:46.695691 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:46.695859 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:46.695865 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:46.695865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:46.695997 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:46.696095 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:46.696180 (+ 85us) server_negotiation.cc:299] Negotiation successful
0927 18:47:46.696257 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:47:46.698379 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3861
2026-09-27T18:47:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1297 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3960](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3960](info): set up 2 sockets
Sep 27 18:47:46 dist-test-slave-vs05 krb5kdc[3960](info): commencing operation
krb5kdc: starting...
W20260927 18:47:47.727264 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.011s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:47 dist-test-slave-vs05 krb5kdc[3960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534867, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:47.845947 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37863
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:47.925357 3973 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:47.925552 3973 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:47.925575 3973 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:47.926628 3973 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:47.926662 3973 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:47.926683 3973 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:47.926697 3973 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:47.926708 3973 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:47.928097 3973 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37863
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3973
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:47.928277 3973 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:47.928434 3973 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:47.933142 3978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:47.933164 3981 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:47.933167 3979 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:47.933341 3973 server_base.cc:1034] running on GCE node
I20260927 18:47:47.933588 3973 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:47.933864 3973 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:47.935007 3973 hybrid_clock.cc:648] HybridClock initialized: now 1790534867934994 us; error 26 us; skew 500 ppm
Sep 27 18:47:47 dist-test-slave-vs05 krb5kdc[3960](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534867, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:47.945603 3973 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:47.946115 3973 webserver.cc:460] Webserver started at http://127.0.0.1:34933/ using document root <none> and password file <none>
I20260927 18:47:47.946341 3973 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:47.946406 3973 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:47.946557 3973 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:47.947701 3973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "c890517ec15b4ebcbaa93a7748037d7d"
format_stamp: "Formatted at 2026-09-27 18:47:47 on dist-test-slave-vs05"
server_key: "4cfc941cc6321902584dc8a79c6359b9"
server_key_iv: "c8328ff0ef909e70ce0d37c9ffeb7aab"
server_key_version: "encryptionkey@0"
I20260927 18:47:47.948074 3973 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "c890517ec15b4ebcbaa93a7748037d7d"
format_stamp: "Formatted at 2026-09-27 18:47:47 on dist-test-slave-vs05"
server_key: "4cfc941cc6321902584dc8a79c6359b9"
server_key_iv: "c8328ff0ef909e70ce0d37c9ffeb7aab"
server_key_version: "encryptionkey@0"
I20260927 18:47:47.949409 3973 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:47.950199 3988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:47.950430 3973 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:47.950558 3973 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "c890517ec15b4ebcbaa93a7748037d7d"
format_stamp: "Formatted at 2026-09-27 18:47:47 on dist-test-slave-vs05"
server_key: "4cfc941cc6321902584dc8a79c6359b9"
server_key_iv: "c8328ff0ef909e70ce0d37c9ffeb7aab"
server_key_version: "encryptionkey@0"
I20260927 18:47:47.950634 3973 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:47.983215 3973 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:47.985232 3973 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:47.985402 3973 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:47.989513 3973 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37863
I20260927 18:47:47.989547 4040 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37863 every 8 connection(s)
I20260927 18:47:47.989912 3973 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:47.990743 4041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:47.991217 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 3973
I20260927 18:47:47.991348 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:47.991561 15803 external_mini_cluster.cc:1428] Setting key 66d6be36ec1833287267e28db6497393
I20260927 18:47:47.993654 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d: Bootstrap starting.
I20260927 18:47:47.994298 4041 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:47.994601 4041 log.cc:824] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:47.995378 4041 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d: No bootstrap required, opened a new log
I20260927 18:47:47.997030 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:47.992070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44697 (local address 127.0.0.1:37863)
0927 18:47:47.992250 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:47.992257 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:47.992282 (+ 25us) server_negotiation.cc:407] Connection header received
0927 18:47:47.992699 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:47.992705 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:47.992751 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:47.992868 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:47.993098 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:47.994049 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:47.995117 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:47.995656 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:47.995739 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:47.996374 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:47.996378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:47.996378 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:47.996547 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:47.996578 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:47.996589 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:47.996638 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:47.996735 (+ 97us) server_negotiation.cc:299] Negotiation successful
0927 18:47:47.996785 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":85,"threads_started":1}
I20260927 18:47:47.997743 4041 raft_consensus.cc:348] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260927 18:47:47.997926 4041 raft_consensus.cc:374] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:47.997967 4041 raft_consensus.cc:729] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c890517ec15b4ebcbaa93a7748037d7d, State: Initialized, Role: FOLLOWER
I20260927 18:47:47.998056 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:47.998127 4041 consensus_queue.cc:260] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260927 18:47:47.998188 4041 raft_consensus.cc:388] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:47.998207 4041 raft_consensus.cc:482] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:47.998234 4041 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:47.998726 4041 raft_consensus.cc:504] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260927 18:47:47.998795 4041 leader_election.cc:302] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c890517ec15b4ebcbaa93a7748037d7d; no voters:
I20260927 18:47:47.998950 4041 leader_election.cc:288] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:47.999094 4050 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:47.999221 4041 sys_catalog.cc:564] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:47.999229 4050 raft_consensus.cc:686] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [term 1 LEADER]: Becoming Leader. State: Replica: c890517ec15b4ebcbaa93a7748037d7d, State: Running, Role: LEADER
I20260927 18:47:47.999480 4050 consensus_queue.cc:237] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } }
I20260927 18:47:47.999861 4050 sys_catalog.cc:455] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader c890517ec15b4ebcbaa93a7748037d7d. Latest consensus state: current_term: 1 leader_uuid: "c890517ec15b4ebcbaa93a7748037d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260927 18:47:47.999939 4050 sys_catalog.cc:458] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:47.999872 4052 sys_catalog.cc:455] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c890517ec15b4ebcbaa93a7748037d7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c890517ec15b4ebcbaa93a7748037d7d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37863 } } }
I20260927 18:47:47.999984 4052 sys_catalog.cc:458] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:48.000119 4060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:48.000674 4060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:48.002247 4060 catalog_manager.cc:1347] Generated new cluster ID: f036e21ac0b54cb09e8593f4d633d0fd
I20260927 18:47:48.002295 4060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:48.004284 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:47.999478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37015 (local address 127.0.0.1:37863)
0927 18:47:47.999512 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:47.999514 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:48.000698 (+ 1184us) server_negotiation.cc:407] Connection header received
0927 18:47:48.000807 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:48.000808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:48.000819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:48.000850 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:48.001153 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:48.001855 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:48.002876 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:48.003156 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:48.003245 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:48.003965 (+ 720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:48.003966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:48.003966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:48.003971 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:48.003985 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:48.003989 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:48.004032 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:48.004191 (+ 159us) server_negotiation.cc:299] Negotiation successful
0927 18:47:48.004212 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:47:48.031565 4060 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:48.032133 4060 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:48.035604 4060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c890517ec15b4ebcbaa93a7748037d7d: Generated new TSK 0
I20260927 18:47:48.035786 4060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:48.039372 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 3973
2026-09-27T18:47:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1341 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:48 dist-test-slave-vs05 krb5kdc[4072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:48 dist-test-slave-vs05 krb5kdc[4072](info): set up 2 sockets
Sep 27 18:47:48 dist-test-slave-vs05 krb5kdc[4072](info): commencing operation
krb5kdc: starting...
W20260927 18:47:49.071336 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:49 dist-test-slave-vs05 krb5kdc[4072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:49.189958 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34681
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:49.270440 4085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:49.270761 4085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:49.270799 4085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:49.272272 4085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:49.272318 4085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:49.272332 4085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:49.272346 4085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:49.272357 4085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:49.274061 4085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:49.274328 4085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:49.274559 4085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:49.279469 4090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:49.279498 4093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:49.279481 4091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:49.279703 4085 server_base.cc:1034] running on GCE node
I20260927 18:47:49.279938 4085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:49.280215 4085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:49.281383 4085 hybrid_clock.cc:648] HybridClock initialized: now 1790534869281365 us; error 34 us; skew 500 ppm
Sep 27 18:47:49 dist-test-slave-vs05 krb5kdc[4072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534869, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:49.283790 4085 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:49.284155 4085 webserver.cc:460] Webserver started at http://127.0.0.1:35383/ using document root <none> and password file <none>
I20260927 18:47:49.284380 4085 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:49.284451 4085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:49.284596 4085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:49.285864 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "f450c118690f46109a20ff505f15b01a"
format_stamp: "Formatted at 2026-09-27 18:47:49 on dist-test-slave-vs05"
server_key: "0c59d3c0d593fec915549fc6f5ede6a8"
server_key_iv: "2d569964d15a01fef4c16be9acf5ad4f"
server_key_version: "encryptionkey@0"
I20260927 18:47:49.286216 4085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "f450c118690f46109a20ff505f15b01a"
format_stamp: "Formatted at 2026-09-27 18:47:49 on dist-test-slave-vs05"
server_key: "0c59d3c0d593fec915549fc6f5ede6a8"
server_key_iv: "2d569964d15a01fef4c16be9acf5ad4f"
server_key_version: "encryptionkey@0"
I20260927 18:47:49.287554 4085 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:49.288235 4100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:49.288406 4085 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:49.288498 4085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "f450c118690f46109a20ff505f15b01a"
format_stamp: "Formatted at 2026-09-27 18:47:49 on dist-test-slave-vs05"
server_key: "0c59d3c0d593fec915549fc6f5ede6a8"
server_key_iv: "2d569964d15a01fef4c16be9acf5ad4f"
server_key_version: "encryptionkey@0"
I20260927 18:47:49.288573 4085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:49.291771 4085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:49.293505 4085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:49.293650 4085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:49.297322 4085 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34681
I20260927 18:47:49.297351 4152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34681 every 8 connection(s)
I20260927 18:47:49.297667 4085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:49.298512 4153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:49.300797 4153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a: Bootstrap starting.
I20260927 18:47:49.301378 4153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:49.301656 4153 log.cc:824] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:49.302294 4153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a: No bootstrap required, opened a new log
I20260927 18:47:49.303997 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4085
I20260927 18:47:49.303978 4153 raft_consensus.cc:348] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260927 18:47:49.304138 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:49.304141 4153 raft_consensus.cc:374] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:49.304214 4153 raft_consensus.cc:729] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f450c118690f46109a20ff505f15b01a, State: Initialized, Role: FOLLOWER
I20260927 18:47:49.304330 15803 external_mini_cluster.cc:1428] Setting key 2673f9eaffb9d4e33f7eb5ecdfc7cc82
I20260927 18:47:49.304385 4153 consensus_queue.cc:260] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260927 18:47:49.304493 4153 raft_consensus.cc:388] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:49.304538 4153 raft_consensus.cc:482] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:49.304587 4153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:49.305497 4153 raft_consensus.cc:504] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260927 18:47:49.305608 4153 leader_election.cc:302] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f450c118690f46109a20ff505f15b01a; no voters:
I20260927 18:47:49.305765 4153 leader_election.cc:288] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:49.305825 4158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:49.305979 4158 raft_consensus.cc:686] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [term 1 LEADER]: Becoming Leader. State: Replica: f450c118690f46109a20ff505f15b01a, State: Running, Role: LEADER
I20260927 18:47:49.306033 4153 sys_catalog.cc:564] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:49.306094 4158 consensus_queue.cc:237] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } }
I20260927 18:47:49.306427 4158 sys_catalog.cc:455] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [sys.catalog]: SysCatalogTable state changed. Reason: New leader f450c118690f46109a20ff505f15b01a. Latest consensus state: current_term: 1 leader_uuid: "f450c118690f46109a20ff505f15b01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } } }
I20260927 18:47:49.306505 4158 sys_catalog.cc:458] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:49.306617 4159 sys_catalog.cc:455] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f450c118690f46109a20ff505f15b01a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f450c118690f46109a20ff505f15b01a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34681 } } }
I20260927 18:47:49.306944 4159 sys_catalog.cc:458] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:49.306715 4166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:49.306896 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:49.304778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38637 (local address 127.0.0.1:34681)
0927 18:47:49.304953 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:49.304963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:49.305010 (+ 47us) server_negotiation.cc:407] Connection header received
0927 18:47:49.305125 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:49.305131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:49.305153 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:49.305274 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:49.305983 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:49.305987 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:49.305988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:49.305993 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:49.306037 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:49.306051 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:49.306108 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:49.306359 (+ 251us) server_negotiation.cc:299] Negotiation successful
0927 18:47:49.306427 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":85,"threads_started":1}
I20260927 18:47:49.307466 4166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:49.308280 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:49.309010 4166 catalog_manager.cc:1347] Generated new cluster ID: 5004d230c4814a46ad231057ba44649a
I20260927 18:47:49.309073 4166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:49.316394 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:49.309788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47037 (local address 127.0.0.1:34681)
0927 18:47:49.309820 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:49.309822 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:49.315384 (+ 5562us) server_negotiation.cc:407] Connection header received
0927 18:47:49.315457 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:49.315458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:49.315479 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:49.315546 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:49.316177 (+ 631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:49.316178 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:49.316179 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:49.316182 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:49.316198 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:49.316201 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:49.316252 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:49.316308 (+ 56us) server_negotiation.cc:299] Negotiation successful
0927 18:47:49.316321 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:47:49.340226 4166 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:49.340436 4166 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:49.346091 4166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f450c118690f46109a20ff505f15b01a: Generated new TSK 0
I20260927 18:47:49.346278 4166 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:49.353755 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4085
2026-09-27T18:47:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1314 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:49 dist-test-slave-vs05 krb5kdc[4184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:49 dist-test-slave-vs05 krb5kdc[4184](info): set up 2 sockets
Sep 27 18:47:49 dist-test-slave-vs05 krb5kdc[4184](info): commencing operation
krb5kdc: starting...
W20260927 18:47:50.407156 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:50.523805 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39009
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39009
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:50.605037 4197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:50.605234 4197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:50.605257 4197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:50.606336 4197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:50.606369 4197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:50.606383 4197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:50.606396 4197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:50.606408 4197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:50.607836 4197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:50.608031 4197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:50.608193 4197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260927 18:47:50.613004 4197 server_base.cc:1034] running on GCE node
W20260927 18:47:50.613018 4203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:50.613018 4202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:50.613130 4205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:50.613493 4197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:50.613760 4197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:50.614915 4197 hybrid_clock.cc:648] HybridClock initialized: now 1790534870614905 us; error 30 us; skew 500 ppm
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534870, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:50.617409 4197 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:50.617851 4197 webserver.cc:460] Webserver started at http://127.0.0.1:44683/ using document root <none> and password file <none>
I20260927 18:47:50.618069 4197 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:50.618149 4197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:50.618297 4197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:50.619387 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "1f047f7ac7f24dc287c3e27aa8568442"
format_stamp: "Formatted at 2026-09-27 18:47:50 on dist-test-slave-vs05"
server_key: "c02362ef31c15dce896590b582995f7a"
server_key_iv: "e100e5830b4e81431dd7f1327de99680"
server_key_version: "encryptionkey@0"
I20260927 18:47:50.619736 4197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "1f047f7ac7f24dc287c3e27aa8568442"
format_stamp: "Formatted at 2026-09-27 18:47:50 on dist-test-slave-vs05"
server_key: "c02362ef31c15dce896590b582995f7a"
server_key_iv: "e100e5830b4e81431dd7f1327de99680"
server_key_version: "encryptionkey@0"
I20260927 18:47:50.621050 4197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:50.621783 4212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:50.621977 4197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:50.622077 4197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "1f047f7ac7f24dc287c3e27aa8568442"
format_stamp: "Formatted at 2026-09-27 18:47:50 on dist-test-slave-vs05"
server_key: "c02362ef31c15dce896590b582995f7a"
server_key_iv: "e100e5830b4e81431dd7f1327de99680"
server_key_version: "encryptionkey@0"
I20260927 18:47:50.622148 4197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:50.635749 4197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:50.637539 4197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:50.637693 4197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:50.641959 4197 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39009
I20260927 18:47:50.641996 4264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39009 every 8 connection(s)
I20260927 18:47:50.642352 4197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:50.643215 4265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:50.645555 4265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442: Bootstrap starting.
I20260927 18:47:50.646126 4265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:50.646407 4265 log.cc:824] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:50.647094 4265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442: No bootstrap required, opened a new log
I20260927 18:47:50.648195 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4197
I20260927 18:47:50.648315 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:50.648488 15803 external_mini_cluster.cc:1428] Setting key ea0948c51beb77e4a34fba9fa8b37550
I20260927 18:47:50.649287 4265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } }
I20260927 18:47:50.649466 4265 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:50.649508 4265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f047f7ac7f24dc287c3e27aa8568442, State: Initialized, Role: FOLLOWER
I20260927 18:47:50.649703 4265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } }
I20260927 18:47:50.649816 4265 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:50.649886 4265 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:50.649938 4265 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:50.650586 4265 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } }
I20260927 18:47:50.650708 4265 leader_election.cc:302] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f047f7ac7f24dc287c3e27aa8568442; no voters:
I20260927 18:47:50.650868 4265 leader_election.cc:288] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:50.650936 4270 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:50.651098 4265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:50.651106 4270 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [term 1 LEADER]: Becoming Leader. State: Replica: 1f047f7ac7f24dc287c3e27aa8568442, State: Running, Role: LEADER
I20260927 18:47:50.651242 4270 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } }
I20260927 18:47:50.651597 4270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f047f7ac7f24dc287c3e27aa8568442. Latest consensus state: current_term: 1 leader_uuid: "1f047f7ac7f24dc287c3e27aa8568442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } } }
I20260927 18:47:50.651604 4271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f047f7ac7f24dc287c3e27aa8568442" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f047f7ac7f24dc287c3e27aa8568442" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39009 } } }
I20260927 18:47:50.651746 4271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:50.651933 4270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:50.652096 4279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:50.652621 4279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:50.654196 4279 catalog_manager.cc:1347] Generated new cluster ID: 5bb7c86e6f4549568f931f10ff1023e9
I20260927 18:47:50.654245 4279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534870, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:47:50.660586 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:50.649015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:42037 (local address 127.0.0.1:39009)
0927 18:47:50.649233 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:50.649242 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:50.649275 (+ 33us) server_negotiation.cc:407] Connection header received
0927 18:47:50.649385 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:50.649392 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:50.649457 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:50.649606 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:50.650319 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:50.651804 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:50.652984 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:50.653521 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:50.653793 (+ 272us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:50.656590 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:50.656594 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:50.656594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:50.656606 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:50.658545 (+ 1939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:50.658972 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:50.658974 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:50.658975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:50.659092 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:50.659348 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:50.659350 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:50.659350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:50.659649 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:50.660186 (+ 537us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:50.660287 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 18:47:50.660389 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260927 18:47:50.661787 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:50.667752 4279 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:50.668427 4279 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:50.670720 4269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:50.663166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:37421 (local address 127.0.0.1:39009)
0927 18:47:50.663221 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:50.663223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:50.663303 (+ 80us) server_negotiation.cc:407] Connection header received
0927 18:47:50.663396 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:50.663396 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:50.663406 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:50.663434 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:50.664119 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:50.664898 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:50.665982 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:50.666342 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:50.666385 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:50.667866 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:50.667868 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:50.667868 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:50.667875 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:50.669455 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:50.669866 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:50.669867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:50.669868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:50.669918 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:50.670134 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:50.670135 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:50.670135 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:50.670263 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:50.670337 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:50.670571 (+ 234us) server_negotiation.cc:299] Negotiation successful
0927 18:47:50.670634 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260927 18:47:50.672317 4279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f047f7ac7f24dc287c3e27aa8568442: Generated new TSK 0
I20260927 18:47:50.672515 4279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:50.677353 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4197
W20260927 18:47:50.681468 4182 connection.cc:535] client connection to 127.0.0.1:39009 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39009): Connection reset by peer (error 104)
2026-09-27T18:47:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4296](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4296](info): set up 2 sockets
Sep 27 18:47:50 dist-test-slave-vs05 krb5kdc[4296](info): commencing operation
krb5kdc: starting...
W20260927 18:47:51.723361 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.028s user 0.001s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:51 dist-test-slave-vs05 krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534871, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:51.848148 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38713
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:51.931686 4309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:51.931892 4309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:51.931915 4309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:51.933053 4309 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:51.933095 4309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:51.933109 4309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:51.933162 4309 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:51.933183 4309 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:51.934623 4309 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38713
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:51.934877 4309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:51.935096 4309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:51.939953 4317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:51.939957 4314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:51.939980 4315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:51.940133 4309 server_base.cc:1034] running on GCE node
I20260927 18:47:51.940419 4309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:51.940665 4309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:51.941836 4309 hybrid_clock.cc:648] HybridClock initialized: now 1790534871941812 us; error 47 us; skew 500 ppm
Sep 27 18:47:51 dist-test-slave-vs05 krb5kdc[4296](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534871, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:51.944419 4309 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:51.944835 4309 webserver.cc:460] Webserver started at http://127.0.0.1:33091/ using document root <none> and password file <none>
I20260927 18:47:51.945050 4309 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:51.945115 4309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:51.945253 4309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:51.946295 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "6ef52e6d624745eaaabb2f1de48da7a1"
format_stamp: "Formatted at 2026-09-27 18:47:51 on dist-test-slave-vs05"
server_key: "570cee48e9e7af5e2e3e08a9a3304c41"
server_key_iv: "6d1d7ac86e3eb4f4860986fc1dde44d5"
server_key_version: "encryptionkey@0"
I20260927 18:47:51.946691 4309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "6ef52e6d624745eaaabb2f1de48da7a1"
format_stamp: "Formatted at 2026-09-27 18:47:51 on dist-test-slave-vs05"
server_key: "570cee48e9e7af5e2e3e08a9a3304c41"
server_key_iv: "6d1d7ac86e3eb4f4860986fc1dde44d5"
server_key_version: "encryptionkey@0"
I20260927 18:47:51.948092 4309 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:47:51.948833 4324 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:51.949046 4309 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:47:51.949150 4309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "6ef52e6d624745eaaabb2f1de48da7a1"
format_stamp: "Formatted at 2026-09-27 18:47:51 on dist-test-slave-vs05"
server_key: "570cee48e9e7af5e2e3e08a9a3304c41"
server_key_iv: "6d1d7ac86e3eb4f4860986fc1dde44d5"
server_key_version: "encryptionkey@0"
I20260927 18:47:51.949221 4309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:51.983865 4309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:51.985853 4309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:51.986016 4309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:51.990095 4309 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38713
I20260927 18:47:51.990134 4376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38713 every 8 connection(s)
I20260927 18:47:51.990509 4309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:51.991463 4377 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:51.993759 4377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1: Bootstrap starting.
I20260927 18:47:51.993988 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4309
I20260927 18:47:51.994108 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:51.994287 15803 external_mini_cluster.cc:1428] Setting key 7d26c462c3cd857404142283891a666b
I20260927 18:47:51.994494 4377 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:51.994848 4377 log.cc:824] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:51.995688 4377 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1: No bootstrap required, opened a new log
I20260927 18:47:51.998024 4377 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260927 18:47:51.998157 4377 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:51.998181 4377 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ef52e6d624745eaaabb2f1de48da7a1, State: Initialized, Role: FOLLOWER
I20260927 18:47:51.998322 4377 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260927 18:47:51.998409 4377 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:51.998446 4377 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:51.998492 4377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:51.999154 4377 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260927 18:47:51.999279 4377 leader_election.cc:302] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6ef52e6d624745eaaabb2f1de48da7a1; no voters:
I20260927 18:47:51.999460 4377 leader_election.cc:288] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:51.999518 4382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:51.999687 4382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [term 1 LEADER]: Becoming Leader. State: Replica: 6ef52e6d624745eaaabb2f1de48da7a1, State: Running, Role: LEADER
I20260927 18:47:51.999749 4377 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:51.999775 4382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } }
I20260927 18:47:52.000653 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:51.994807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:35005 (local address 127.0.0.1:38713)
0927 18:47:51.994982 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:51.994991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:51.995030 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:47:51.995682 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:51.995691 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:51.995765 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:51.995939 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:51.996155 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:51.997561 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:51.998378 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:51.998993 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:51.999111 (+ 118us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:51.999856 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:51.999860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:51.999860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:52.000059 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:52.000101 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:52.000116 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:52.000198 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:52.000419 (+ 221us) server_negotiation.cc:299] Negotiation successful
0927 18:47:52.000447 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260927 18:47:52.000931 4384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6ef52e6d624745eaaabb2f1de48da7a1. Latest consensus state: current_term: 1 leader_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } } }
I20260927 18:47:52.001031 4384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:52.001214 4383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6ef52e6d624745eaaabb2f1de48da7a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38713 } } }
I20260927 18:47:52.001345 4397 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:52.001437 4383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:52.001693 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:52.001890 4397 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:52.003600 4397 catalog_manager.cc:1347] Generated new cluster ID: 58d23a2de33a463e875fd461058fc3e5
I20260927 18:47:52.003655 4397 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:52.008630 4380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:52.003279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41011 (local address 127.0.0.1:38713)
0927 18:47:52.003311 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:52.003313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:52.005849 (+ 2536us) server_negotiation.cc:407] Connection header received
0927 18:47:52.005924 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:52.005925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:52.005936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:52.005991 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:52.006140 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:52.006619 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:52.007437 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:52.007662 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:52.007746 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:52.008409 (+ 663us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:52.008410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:52.008410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:52.008415 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:52.008429 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:52.008433 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:52.008487 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:52.008545 (+ 58us) server_negotiation.cc:299] Negotiation successful
0927 18:47:52.008553 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:47:52.016741 4397 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:52.017328 4397 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:52.022954 4397 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6ef52e6d624745eaaabb2f1de48da7a1: Generated new TSK 0
I20260927 18:47:52.023145 4397 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:52.028873 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4309
W20260927 18:47:52.033123 4294 connection.cc:535] client connection to 127.0.0.1:38713 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38713): Connection reset by peer (error 104)
2026-09-27T18:47:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1351 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:52 dist-test-slave-vs05 krb5kdc[4408](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:52 dist-test-slave-vs05 krb5kdc[4408](info): set up 2 sockets
Sep 27 18:47:52 dist-test-slave-vs05 krb5kdc[4408](info): commencing operation
krb5kdc: starting...
W20260927 18:47:53.071409 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:53 dist-test-slave-vs05 krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534873, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:53.189638 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41085
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41085
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:53.269320 4421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:53.269547 4421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:53.269586 4421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:53.270748 4421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:53.270814 4421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:53.270839 4421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:53.270870 4421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:53.270905 4421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:53.272392 4421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:53.272653 4421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:53.272894 4421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:53.277763 4429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:53.277869 4421 server_base.cc:1034] running on GCE node
W20260927 18:47:53.277801 4427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:53.277791 4426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:53.278287 4421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:53.278540 4421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:53.279680 4421 hybrid_clock.cc:648] HybridClock initialized: now 1790534873279636 us; error 51 us; skew 500 ppm
Sep 27 18:47:53 dist-test-slave-vs05 krb5kdc[4408](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534873, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:53.286151 4421 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:53.286546 4421 webserver.cc:460] Webserver started at http://127.0.0.1:35609/ using document root <none> and password file <none>
I20260927 18:47:53.286728 4421 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:53.286767 4421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:53.286854 4421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:53.287940 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "42a0ccf1e88e47b79677494b2da6a491"
format_stamp: "Formatted at 2026-09-27 18:47:53 on dist-test-slave-vs05"
server_key: "32c38b7e104a9d5c11188b5520523842"
server_key_iv: "b3a44725ae275d17b3b7537dcfd7304f"
server_key_version: "encryptionkey@0"
I20260927 18:47:53.288244 4421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "42a0ccf1e88e47b79677494b2da6a491"
format_stamp: "Formatted at 2026-09-27 18:47:53 on dist-test-slave-vs05"
server_key: "32c38b7e104a9d5c11188b5520523842"
server_key_iv: "b3a44725ae275d17b3b7537dcfd7304f"
server_key_version: "encryptionkey@0"
I20260927 18:47:53.289532 4421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:53.290233 4436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:53.290431 4421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:53.290518 4421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "42a0ccf1e88e47b79677494b2da6a491"
format_stamp: "Formatted at 2026-09-27 18:47:53 on dist-test-slave-vs05"
server_key: "32c38b7e104a9d5c11188b5520523842"
server_key_iv: "b3a44725ae275d17b3b7537dcfd7304f"
server_key_version: "encryptionkey@0"
I20260927 18:47:53.290575 4421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:53.293831 4421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:53.296200 4421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:53.296361 4421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:53.300104 4421 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41085
I20260927 18:47:53.300136 4488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41085 every 8 connection(s)
I20260927 18:47:53.300470 4421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:53.301340 4489 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:53.303781 4489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491: Bootstrap starting.
I20260927 18:47:53.304203 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4421
I20260927 18:47:53.304320 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:53.304508 15803 external_mini_cluster.cc:1428] Setting key 18e9a1543a60b7763b32a17f0a781268
I20260927 18:47:53.304543 4489 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:53.304828 4489 log.cc:824] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:53.305575 4489 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491: No bootstrap required, opened a new log
I20260927 18:47:53.307137 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:53.305078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:57535 (local address 127.0.0.1:41085)
0927 18:47:53.305268 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:53.305279 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:53.305310 (+ 31us) server_negotiation.cc:407] Connection header received
0927 18:47:53.305810 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:53.305817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:53.305843 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:53.306003 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:53.306697 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:53.306701 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:53.306701 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:53.306705 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:53.306739 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:53.306749 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:53.306841 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:53.306879 (+ 38us) server_negotiation.cc:299] Negotiation successful
0927 18:47:53.306945 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
I20260927 18:47:53.308065 4489 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } }
I20260927 18:47:53.308259 4489 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:53.308285 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:53.308297 4489 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42a0ccf1e88e47b79677494b2da6a491, State: Initialized, Role: FOLLOWER
I20260927 18:47:53.308414 4489 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } }
I20260927 18:47:53.308496 4489 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:53.308519 4489 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:53.308588 4489 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:53.309126 4489 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } }
I20260927 18:47:53.309216 4489 leader_election.cc:302] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42a0ccf1e88e47b79677494b2da6a491; no voters:
I20260927 18:47:53.309409 4489 leader_election.cc:288] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:53.309428 4497 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:53.309607 4497 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [term 1 LEADER]: Becoming Leader. State: Replica: 42a0ccf1e88e47b79677494b2da6a491, State: Running, Role: LEADER
I20260927 18:47:53.309666 4489 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:53.309724 4497 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } }
I20260927 18:47:53.310055 4497 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42a0ccf1e88e47b79677494b2da6a491. Latest consensus state: current_term: 1 leader_uuid: "42a0ccf1e88e47b79677494b2da6a491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } } }
I20260927 18:47:53.310065 4499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42a0ccf1e88e47b79677494b2da6a491" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42a0ccf1e88e47b79677494b2da6a491" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41085 } } }
I20260927 18:47:53.310153 4497 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:53.310181 4499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:53.310511 4506 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:53.311124 4493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:53.309837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:56359 (local address 127.0.0.1:41085)
0927 18:47:53.309869 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:53.309871 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:53.309974 (+ 103us) server_negotiation.cc:407] Connection header received
0927 18:47:53.310069 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:53.310070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:53.310075 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:53.310105 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:53.310822 (+ 717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:53.310824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:53.310824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:53.310828 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:53.310846 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:53.310851 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:53.310921 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:53.311050 (+ 129us) server_negotiation.cc:299] Negotiation successful
0927 18:47:53.311064 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260927 18:47:53.311597 4506 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:53.313256 4506 catalog_manager.cc:1347] Generated new cluster ID: f54cb49de86b43dd961b67ac22b74b76
I20260927 18:47:53.313325 4506 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:53.338330 4506 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:53.338554 4506 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:53.342381 4506 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42a0ccf1e88e47b79677494b2da6a491: Generated new TSK 0
I20260927 18:47:53.342598 4506 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:53.351487 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4421
2026-09-27T18:47:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1323 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:53 dist-test-slave-vs05 krb5kdc[4520](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:53 dist-test-slave-vs05 krb5kdc[4520](info): set up 2 sockets
Sep 27 18:47:53 dist-test-slave-vs05 krb5kdc[4520](info): commencing operation
krb5kdc: starting...
W20260927 18:47:54.387527 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:54.506508 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38191
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38191
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:54.586647 4533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:54.586884 4533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:54.586925 4533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:54.588157 4533 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:54.588232 4533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:54.588256 4533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:54.588291 4533 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:54.588322 4533 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:54.589793 4533 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38191
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38191
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:54.590071 4533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:54.590305 4533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:54.595153 4538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:54.595317 4533 server_base.cc:1034] running on GCE node
W20260927 18:47:54.595203 4539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:54.595188 4541 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:54.595670 4533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:54.595957 4533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:54.597122 4533 hybrid_clock.cc:648] HybridClock initialized: now 1790534874597100 us; error 36 us; skew 500 ppm
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4520](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534874, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:54.599740 4533 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:54.600245 4533 webserver.cc:460] Webserver started at http://127.0.0.1:38105/ using document root <none> and password file <none>
I20260927 18:47:54.600468 4533 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:54.600512 4533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:54.600654 4533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:54.601701 4533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "72111ddd8a104d4db217f3f1834106b1"
format_stamp: "Formatted at 2026-09-27 18:47:54 on dist-test-slave-vs05"
server_key: "da9941081ed6a4e0392cc352915a7895"
server_key_iv: "85e1e8b730a008347c9f4a92a03754b2"
server_key_version: "encryptionkey@0"
I20260927 18:47:54.602058 4533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "72111ddd8a104d4db217f3f1834106b1"
format_stamp: "Formatted at 2026-09-27 18:47:54 on dist-test-slave-vs05"
server_key: "da9941081ed6a4e0392cc352915a7895"
server_key_iv: "85e1e8b730a008347c9f4a92a03754b2"
server_key_version: "encryptionkey@0"
I20260927 18:47:54.603423 4533 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:47:54.604123 4548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:54.604313 4533 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:54.604403 4533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "72111ddd8a104d4db217f3f1834106b1"
format_stamp: "Formatted at 2026-09-27 18:47:54 on dist-test-slave-vs05"
server_key: "da9941081ed6a4e0392cc352915a7895"
server_key_iv: "85e1e8b730a008347c9f4a92a03754b2"
server_key_version: "encryptionkey@0"
I20260927 18:47:54.604475 4533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:54.617045 4533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:54.619367 4533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:54.619529 4533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:54.623464 4533 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38191
I20260927 18:47:54.623526 4600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38191 every 8 connection(s)
I20260927 18:47:54.623885 4533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:54.624719 4601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:54.627029 4601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: Bootstrap starting.
I20260927 18:47:54.627677 4601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:54.628077 4601 log.cc:824] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:54.628749 4601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: No bootstrap required, opened a new log
I20260927 18:47:54.630292 4601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260927 18:47:54.630462 4601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:54.630511 4601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72111ddd8a104d4db217f3f1834106b1, State: Initialized, Role: FOLLOWER
I20260927 18:47:54.630663 4601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260927 18:47:54.630755 4601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:54.630800 4601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:54.630856 4601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:54.631076 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4533
I20260927 18:47:54.631208 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:54.631389 15803 external_mini_cluster.cc:1428] Setting key f0b36b2234fc8eca1306e978bb7052bf
I20260927 18:47:54.631533 4601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260927 18:47:54.631642 4601 leader_election.cc:302] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 72111ddd8a104d4db217f3f1834106b1; no voters:
I20260927 18:47:54.631815 4601 leader_election.cc:288] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:54.632017 4604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:54.632090 4601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:54.632130 4604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [term 1 LEADER]: Becoming Leader. State: Replica: 72111ddd8a104d4db217f3f1834106b1, State: Running, Role: LEADER
I20260927 18:47:54.632208 4604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } }
I20260927 18:47:54.632540 4608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 72111ddd8a104d4db217f3f1834106b1. Latest consensus state: current_term: 1 leader_uuid: "72111ddd8a104d4db217f3f1834106b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } } }
I20260927 18:47:54.632638 4608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:54.632658 4606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "72111ddd8a104d4db217f3f1834106b1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "72111ddd8a104d4db217f3f1834106b1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38191 } } }
I20260927 18:47:54.632751 4606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1 [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:54.633872 4621 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:54.633931 4621 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:54.634017 4614 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:54.634191 4614 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:54.636281 4614 catalog_manager.cc:1347] Generated new cluster ID: 3996e37f5c5f4c9e833299ef79330c8c
I20260927 18:47:54.636334 4614 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4520](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534874, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260927 18:47:54.642155 4614 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:54.642877 4614 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:54.644054 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:54.631863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41581 (local address 127.0.0.1:38191)
0927 18:47:54.632023 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:54.632034 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:54.632087 (+ 53us) server_negotiation.cc:407] Connection header received
0927 18:47:54.632431 (+ 344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:54.632438 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:54.632495 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:54.632621 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:54.633775 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:54.635067 (+ 1292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:54.635931 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:54.636538 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:54.636637 (+ 99us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:54.639495 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:54.639499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:54.639500 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:54.639512 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:54.642197 (+ 2685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:54.642738 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:54.642741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:54.642742 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:54.642862 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:54.643136 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:54.643137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:54.643137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:54.643457 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:54.643638 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:54.643726 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 18:47:54.643835 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":70,"threads_started":1}
I20260927 18:47:54.645004 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:54.652501 4614 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 72111ddd8a104d4db217f3f1834106b1: Generated new TSK 0
I20260927 18:47:54.652695 4614 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:54.653772 4607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:54.646424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:35761 (local address 127.0.0.1:38191)
0927 18:47:54.646460 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:54.646462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:54.646514 (+ 52us) server_negotiation.cc:407] Connection header received
0927 18:47:54.646569 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:54.646570 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:54.646593 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:54.646662 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:54.647313 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:54.648128 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:54.649143 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:54.649453 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:54.649508 (+ 55us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:54.650933 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:54.650934 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:54.650934 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:54.650940 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:54.652636 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:54.653081 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:54.653082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:54.653082 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:54.653132 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:54.653350 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:54.653351 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:54.653351 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:54.653443 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:54.653519 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:54.653668 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:47:54.653715 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:47:54.656324 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4533
W20260927 18:47:54.661013 4518 connection.cc:535] client connection to 127.0.0.1:38191 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38191): Connection reset by peer (error 104)
2026-09-27T18:47:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4632](info): set up 2 sockets
Sep 27 18:47:54 dist-test-slave-vs05 krb5kdc[4632](info): commencing operation
krb5kdc: starting...
W20260927 18:47:55.691469 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.017s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:55 dist-test-slave-vs05 krb5kdc[4632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:55.810292 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40175
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:55.891045 4645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:55.891347 4645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:55.891388 4645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:55.892524 4645 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:55.892596 4645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:55.892620 4645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:55.892640 4645 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:55.892670 4645 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:55.894146 4645 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40175
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:55.894420 4645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:55.894646 4645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:55.899556 4650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:55.899576 4653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:55.899571 4651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:55.899824 4645 server_base.cc:1034] running on GCE node
I20260927 18:47:55.900028 4645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:55.900259 4645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:55.901401 4645 hybrid_clock.cc:648] HybridClock initialized: now 1790534875901389 us; error 30 us; skew 500 ppm
Sep 27 18:47:55 dist-test-slave-vs05 krb5kdc[4632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534875, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:55.904052 4645 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:55.904544 4645 webserver.cc:460] Webserver started at http://127.0.0.1:36707/ using document root <none> and password file <none>
I20260927 18:47:55.904785 4645 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:55.904831 4645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:55.905006 4645 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:55.906131 4645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "4c0b613d496541bfbe35c0c552c03ddf"
format_stamp: "Formatted at 2026-09-27 18:47:55 on dist-test-slave-vs05"
server_key: "9135de3e927638588c037678e945c5a4"
server_key_iv: "9ddad6240944d9e6da3fea1f91f4c141"
server_key_version: "encryptionkey@0"
I20260927 18:47:55.906491 4645 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "4c0b613d496541bfbe35c0c552c03ddf"
format_stamp: "Formatted at 2026-09-27 18:47:55 on dist-test-slave-vs05"
server_key: "9135de3e927638588c037678e945c5a4"
server_key_iv: "9ddad6240944d9e6da3fea1f91f4c141"
server_key_version: "encryptionkey@0"
I20260927 18:47:55.907958 4645 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:47:55.908764 4660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:55.908974 4645 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:47:55.909087 4645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "4c0b613d496541bfbe35c0c552c03ddf"
format_stamp: "Formatted at 2026-09-27 18:47:55 on dist-test-slave-vs05"
server_key: "9135de3e927638588c037678e945c5a4"
server_key_iv: "9ddad6240944d9e6da3fea1f91f4c141"
server_key_version: "encryptionkey@0"
I20260927 18:47:55.909170 4645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:55.925616 4645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:55.927520 4645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:55.927685 4645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:55.931840 4645 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40175
I20260927 18:47:55.931877 4712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40175 every 8 connection(s)
I20260927 18:47:55.932230 4645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:55.933063 4713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:55.934866 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4645
I20260927 18:47:55.934966 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:55.935237 15803 external_mini_cluster.cc:1428] Setting key bb1ff414b85c1272a6295c52c36fef8e
I20260927 18:47:55.935714 4713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: Bootstrap starting.
I20260927 18:47:55.936448 4713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:55.936770 4713 log.cc:824] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:55.937580 4713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: No bootstrap required, opened a new log
I20260927 18:47:55.939841 4713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260927 18:47:55.940007 4713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:55.940039 4713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c0b613d496541bfbe35c0c552c03ddf, State: Initialized, Role: FOLLOWER
I20260927 18:47:55.940171 4713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260927 18:47:55.940251 4713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:55.940281 4713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:55.940341 4713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:55.941071 4713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260927 18:47:55.941187 4713 leader_election.cc:302] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c0b613d496541bfbe35c0c552c03ddf; no voters:
I20260927 18:47:55.941354 4713 leader_election.cc:288] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:55.941388 4718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:55.941568 4718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [term 1 LEADER]: Becoming Leader. State: Replica: 4c0b613d496541bfbe35c0c552c03ddf, State: Running, Role: LEADER
I20260927 18:47:55.941627 4713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:55.941861 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:55.935875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:55665 (local address 127.0.0.1:40175)
0927 18:47:55.936082 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:55.936092 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:55.936131 (+ 39us) server_negotiation.cc:407] Connection header received
0927 18:47:55.936749 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:55.936758 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:55.936838 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:55.936996 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:55.937270 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:55.938627 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:55.939486 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:55.940116 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:55.940221 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:55.940979 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:55.940983 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:55.940983 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:55.941194 (+ 211us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:55.941242 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:55.941257 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:55.941350 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:55.941602 (+ 252us) server_negotiation.cc:299] Negotiation successful
0927 18:47:55.941630 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":96,"threads_started":1}
I20260927 18:47:55.941725 4718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } }
I20260927 18:47:55.942953 4719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c0b613d496541bfbe35c0c552c03ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } } }
I20260927 18:47:55.942955 4720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c0b613d496541bfbe35c0c552c03ddf. Latest consensus state: current_term: 1 leader_uuid: "4c0b613d496541bfbe35c0c552c03ddf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c0b613d496541bfbe35c0c552c03ddf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40175 } } }
I20260927 18:47:55.943248 4719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:55.943254 4720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:55.943310 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260927 18:47:55.944187 4732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:55.944242 4732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:55.944293 4733 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:55.944484 4733 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:55.946015 4733 catalog_manager.cc:1347] Generated new cluster ID: fccfeb8b80df405c8b4768ae455ebc66
I20260927 18:47:55.946076 4733 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:55.948309 4716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:55.944760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:35411 (local address 127.0.0.1:40175)
0927 18:47:55.944794 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:55.944796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:55.945081 (+ 285us) server_negotiation.cc:407] Connection header received
0927 18:47:55.945127 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:55.945128 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:55.945138 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:55.945196 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:55.945326 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:55.945961 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:55.946981 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:55.947307 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:55.947342 (+ 35us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:55.948072 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:55.948073 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:55.948073 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:55.948078 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:55.948091 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:55.948096 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:55.948124 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:55.948231 (+ 107us) server_negotiation.cc:299] Negotiation successful
0927 18:47:55.948240 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:47:55.954998 4733 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:55.955761 4733 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:55.966918 4733 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c0b613d496541bfbe35c0c552c03ddf: Generated new TSK 0
I20260927 18:47:55.967146 4733 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:55.972615 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4645
W20260927 18:47:55.976959 4630 connection.cc:535] client connection to 127.0.0.1:40175 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40175): Connection reset by peer (error 104)
2026-09-27T18:47:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1315 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:56 dist-test-slave-vs05 krb5kdc[4744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:56 dist-test-slave-vs05 krb5kdc[4744](info): set up 2 sockets
Sep 27 18:47:56 dist-test-slave-vs05 krb5kdc[4744](info): commencing operation
krb5kdc: starting...
W20260927 18:47:57.003290 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.013s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:57 dist-test-slave-vs05 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260927 18:47:57.121336 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39599
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:57.200521 4757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:57.200762 4757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:57.200798 4757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:57.201936 4757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:57.202004 4757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:57.202028 4757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:57.202049 4757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:57.202075 4757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:57.203562 4757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39599
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:57.203835 4757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:57.204063 4757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:57.208797 4762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:57.208832 4763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:57.209072 4757 server_base.cc:1034] running on GCE node
W20260927 18:47:57.208848 4765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:57.209381 4757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:57.209666 4757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:57.210824 4757 hybrid_clock.cc:648] HybridClock initialized: now 1790534877210800 us; error 37 us; skew 500 ppm
Sep 27 18:47:57 dist-test-slave-vs05 krb5kdc[4744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534877, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:57.213164 4757 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260927 18:47:57.213616 4757 webserver.cc:460] Webserver started at http://127.0.0.1:37115/ using document root <none> and password file <none>
I20260927 18:47:57.213837 4757 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:57.213910 4757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:57.214059 4757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:57.215078 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "6296f04f080245bcaf4d32ef8f7c17d2"
format_stamp: "Formatted at 2026-09-27 18:47:57 on dist-test-slave-vs05"
server_key: "fc0c25e34601ec782445ede53974df6a"
server_key_iv: "2dcd25dea273fc62b0f1cbf91ece6873"
server_key_version: "encryptionkey@0"
I20260927 18:47:57.215502 4757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "6296f04f080245bcaf4d32ef8f7c17d2"
format_stamp: "Formatted at 2026-09-27 18:47:57 on dist-test-slave-vs05"
server_key: "fc0c25e34601ec782445ede53974df6a"
server_key_iv: "2dcd25dea273fc62b0f1cbf91ece6873"
server_key_version: "encryptionkey@0"
I20260927 18:47:57.216766 4757 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260927 18:47:57.217433 4772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:57.217602 4757 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:57.217705 4757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "6296f04f080245bcaf4d32ef8f7c17d2"
format_stamp: "Formatted at 2026-09-27 18:47:57 on dist-test-slave-vs05"
server_key: "fc0c25e34601ec782445ede53974df6a"
server_key_iv: "2dcd25dea273fc62b0f1cbf91ece6873"
server_key_version: "encryptionkey@0"
I20260927 18:47:57.217779 4757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:57.221001 4757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:57.222718 4757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:57.222864 4757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:57.226819 4757 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39599
I20260927 18:47:57.226866 4824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39599 every 8 connection(s)
I20260927 18:47:57.227221 4757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:57.228003 4825 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:57.230051 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2: Bootstrap starting.
I20260927 18:47:57.230551 4825 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:57.230770 4825 log.cc:824] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:57.231432 4825 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2: No bootstrap required, opened a new log
I20260927 18:47:57.232895 4825 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } }
I20260927 18:47:57.233022 4825 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:57.233043 4825 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6296f04f080245bcaf4d32ef8f7c17d2, State: Initialized, Role: FOLLOWER
I20260927 18:47:57.233136 4825 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } }
I20260927 18:47:57.233183 4825 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:57.233204 4825 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:57.233228 4825 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:57.233682 4825 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } }
I20260927 18:47:57.233759 4825 leader_election.cc:302] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6296f04f080245bcaf4d32ef8f7c17d2; no voters:
I20260927 18:47:57.233871 4825 leader_election.cc:288] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:57.233924 4828 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:57.234088 4828 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [term 1 LEADER]: Becoming Leader. State: Replica: 6296f04f080245bcaf4d32ef8f7c17d2, State: Running, Role: LEADER
I20260927 18:47:57.234203 4828 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } }
I20260927 18:47:57.234139 4825 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:57.234556 4830 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6296f04f080245bcaf4d32ef8f7c17d2. Latest consensus state: current_term: 1 leader_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } } }
I20260927 18:47:57.234639 4830 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:57.234679 4829 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6296f04f080245bcaf4d32ef8f7c17d2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39599 } } }
I20260927 18:47:57.234738 4829 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:57.234848 4833 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:57.235603 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4757
I20260927 18:47:57.235627 4833 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:57.235718 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:57.235936 15803 external_mini_cluster.cc:1428] Setting key d6260fc96c2bc6520e6fc7cf135ef540
I20260927 18:47:57.237561 4833 catalog_manager.cc:1347] Generated new cluster ID: 26ee7cbadb7a46d0b875662adc8209da
I20260927 18:47:57.237643 4833 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:57.238029 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:57.236476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:59465 (local address 127.0.0.1:39599)
0927 18:47:57.236639 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:57.236648 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:57.236677 (+ 29us) server_negotiation.cc:407] Connection header received
0927 18:47:57.236787 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:57.236793 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:57.236824 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:57.236958 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:57.237619 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:57.237622 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:57.237623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:57.237628 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:57.237669 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:57.237683 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:57.237751 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:57.237820 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:47:57.237863 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260927 18:47:57.239009 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:57.241600 4847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:57.240618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:54349 (local address 127.0.0.1:39599)
0927 18:47:57.240654 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:57.240656 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:57.240722 (+ 66us) server_negotiation.cc:407] Connection header received
0927 18:47:57.240796 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:57.240797 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:57.240801 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:57.240844 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:57.241405 (+ 561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:57.241406 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:57.241406 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:57.241410 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:57.241423 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:57.241426 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:57.241474 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:57.241531 (+ 57us) server_negotiation.cc:299] Negotiation successful
0927 18:47:57.241544 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:47:57.259990 4833 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:57.260159 4833 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:57.262068 4833 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6296f04f080245bcaf4d32ef8f7c17d2: Generated new TSK 0
I20260927 18:47:57.262226 4833 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:57.264144 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4757
2026-09-27T18:47:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1291 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:57 dist-test-slave-vs05 krb5kdc[4856](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:57 dist-test-slave-vs05 krb5kdc[4856](info): set up 2 sockets
Sep 27 18:47:57 dist-test-slave-vs05 krb5kdc[4856](info): commencing operation
krb5kdc: starting...
W20260927 18:47:58.291347 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.010s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:47:58.409567 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:37271
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:39397
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:58.489923 4869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:58.490119 4869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:58.490144 4869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:58.491326 4869 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:58.491366 4869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:58.491380 4869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:58.491393 4869 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:58.491432 4869 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:58.492889 4869 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:39397
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:37271
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:58.493132 4869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:58.493361 4869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:58.498205 4874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:58.498234 4875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:58.498286 4877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:58.498515 4869 server_base.cc:1034] running on GCE node
I20260927 18:47:58.498795 4869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:58.499076 4869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:58.500238 4869 hybrid_clock.cc:648] HybridClock initialized: now 1790534878500223 us; error 45 us; skew 500 ppm
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4856](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534878, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:58.509809 4869 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:58.510340 4869 webserver.cc:460] Webserver started at http://127.15.110.254:40465/ using document root <none> and password file <none>
I20260927 18:47:58.510591 4869 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:58.510659 4869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:58.510805 4869 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:58.511947 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "598ba3d9882045e2bb31a612de5bee8f"
format_stamp: "Formatted at 2026-09-27 18:47:58 on dist-test-slave-vs05"
server_key: "b864b8d690c4412a3969ccbe3f349262"
server_key_iv: "5186b7e362778db2f2639bac7c8248ec"
server_key_version: "encryptionkey@0"
I20260927 18:47:58.512302 4869 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "598ba3d9882045e2bb31a612de5bee8f"
format_stamp: "Formatted at 2026-09-27 18:47:58 on dist-test-slave-vs05"
server_key: "b864b8d690c4412a3969ccbe3f349262"
server_key_iv: "5186b7e362778db2f2639bac7c8248ec"
server_key_version: "encryptionkey@0"
I20260927 18:47:58.513633 4869 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:58.514406 4884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:58.514581 4869 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:58.514685 4869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "598ba3d9882045e2bb31a612de5bee8f"
format_stamp: "Formatted at 2026-09-27 18:47:58 on dist-test-slave-vs05"
server_key: "b864b8d690c4412a3969ccbe3f349262"
server_key_iv: "5186b7e362778db2f2639bac7c8248ec"
server_key_version: "encryptionkey@0"
I20260927 18:47:58.514760 4869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:58.521787 4869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:58.524037 4869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:58.524199 4869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:58.528060 4869 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:37271
I20260927 18:47:58.528091 4936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:37271 every 8 connection(s)
I20260927 18:47:58.528421 4869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:58.529213 4937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:58.531391 4937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: Bootstrap starting.
I20260927 18:47:58.531896 4937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:58.532119 4937 log.cc:824] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:58.532728 4937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: No bootstrap required, opened a new log
I20260927 18:47:58.534219 4937 raft_consensus.cc:348] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } }
I20260927 18:47:58.534366 4937 raft_consensus.cc:374] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:58.534389 4937 raft_consensus.cc:729] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 598ba3d9882045e2bb31a612de5bee8f, State: Initialized, Role: FOLLOWER
I20260927 18:47:58.534452 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4869
I20260927 18:47:58.534579 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:58.534536 4937 consensus_queue.cc:260] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } }
I20260927 18:47:58.534612 4937 raft_consensus.cc:388] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:58.534636 4937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:58.534703 4937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:58.534788 15803 external_mini_cluster.cc:1428] Setting key 924e92fcbaee6b001343e694151eb848
I20260927 18:47:58.535789 4937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } }
I20260927 18:47:58.535917 4937 leader_election.cc:302] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 598ba3d9882045e2bb31a612de5bee8f; no voters:
I20260927 18:47:58.536078 4937 leader_election.cc:288] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:58.536123 4942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:58.536314 4942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [term 1 LEADER]: Becoming Leader. State: Replica: 598ba3d9882045e2bb31a612de5bee8f, State: Running, Role: LEADER
I20260927 18:47:58.536350 4937 sys_catalog.cc:564] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:58.536417 4942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } }
I20260927 18:47:58.538120 4943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 598ba3d9882045e2bb31a612de5bee8f. Latest consensus state: current_term: 1 leader_uuid: "598ba3d9882045e2bb31a612de5bee8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } } }
I20260927 18:47:58.538129 4944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "598ba3d9882045e2bb31a612de5bee8f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "598ba3d9882045e2bb31a612de5bee8f" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 37271 } } }
I20260927 18:47:58.538234 4943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:58.538244 4944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:58.539541 4958 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:58.539592 4958 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:58.539635 4956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:58.539762 4956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:58.541299 4956 catalog_manager.cc:1347] Generated new cluster ID: 5d516bb6a9874b2ba86eb14cb805fe3d
I20260927 18:47:58.541360 4956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4856](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534878, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:47:58.545400 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:58.535294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48331 (local address 127.15.110.254:37271)
0927 18:47:58.535482 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:58.535490 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:58.535516 (+ 26us) server_negotiation.cc:407] Connection header received
0927 18:47:58.535601 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:58.535606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:58.535643 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:58.535747 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:58.536581 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:58.537793 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:58.538846 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:58.539417 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:58.539547 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:58.542168 (+ 2621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:58.542172 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:58.542172 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:58.542183 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:58.543890 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:58.544263 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:58.544266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:58.544266 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:58.544367 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:58.544593 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:58.544595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:58.544595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:58.544836 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:58.545023 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:58.545107 (+ 84us) server_negotiation.cc:299] Negotiation successful
0927 18:47:58.545210 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":108,"threads_started":1}
I20260927 18:47:58.546356 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:58.557099 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:58.547668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52275 (local address 127.15.110.254:37271)
0927 18:47:58.547709 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:58.547712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:58.551341 (+ 3629us) server_negotiation.cc:407] Connection header received
0927 18:47:58.551418 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:58.551419 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:58.551428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:58.551475 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:58.551947 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:58.552424 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:58.553121 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:58.553347 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:58.553439 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:58.554560 (+ 1121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:58.554561 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:58.554562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:47:58.554568 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:58.555943 (+ 1375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:58.556295 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:58.556296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:58.556297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:58.556349 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:47:58.556572 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:47:58.556573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:47:58.556573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:47:58.556683 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:58.556783 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:58.556925 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 18:47:58.556984 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:47:58.565572 4956 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:58.566068 4956 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:58.568425 4956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 598ba3d9882045e2bb31a612de5bee8f: Generated new TSK 0
I20260927 18:47:58.568596 4956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:58.577294 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4869
W20260927 18:47:58.581583 4854 connection.cc:535] client connection to 127.15.110.254:37271 recv error: Network error: failed to read from TLS socket (remote: 127.15.110.254:37271): Connection reset by peer (error 104)
2026-09-27T18:47:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4968](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4968](info): set up 2 sockets
Sep 27 18:47:58 dist-test-slave-vs05 krb5kdc[4968](info): commencing operation
krb5kdc: starting...
W20260927 18:47:59.611411 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.016s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:47:59 dist-test-slave-vs05 krb5kdc[4968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:47:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:47:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:47:59.728355 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43197
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:35857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:43197
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:47:59.808295 4981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:47:59.808499 4981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:47:59.808522 4981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:47:59.809692 4981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:47:59.809733 4981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:47:59.809748 4981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:47:59.809760 4981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:47:59.809773 4981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:47:59.811319 4981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:35857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:43197
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:43197
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:47:59.811569 4981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:47:59.811805 4981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:47:59.816538 4989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:59.816517 4986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:47:59.816530 4987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:47:59.816952 4981 server_base.cc:1034] running on GCE node
I20260927 18:47:59.817202 4981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:47:59.817495 4981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:47:59.818637 4981 hybrid_clock.cc:648] HybridClock initialized: now 1790534879818604 us; error 60 us; skew 500 ppm
Sep 27 18:47:59 dist-test-slave-vs05 krb5kdc[4968](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534879, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:47:59.821278 4981 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:47:59.821868 4981 webserver.cc:460] Webserver started at http://127.15.110.254:43137/ using document root <none> and password file <none>
I20260927 18:47:59.822113 4981 fs_manager.cc:359] Metadata directory not provided
I20260927 18:47:59.822181 4981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:47:59.822320 4981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:47:59.823463 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "4120f24d88e84db78416300c24311c23"
format_stamp: "Formatted at 2026-09-27 18:47:59 on dist-test-slave-vs05"
server_key: "0c01e627cfa2d58d1abdcf1dc5580202"
server_key_iv: "a1e5bf2ab53fbd957b866e4b451ffb41"
server_key_version: "encryptionkey@0"
I20260927 18:47:59.823827 4981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "4120f24d88e84db78416300c24311c23"
format_stamp: "Formatted at 2026-09-27 18:47:59 on dist-test-slave-vs05"
server_key: "0c01e627cfa2d58d1abdcf1dc5580202"
server_key_iv: "a1e5bf2ab53fbd957b866e4b451ffb41"
server_key_version: "encryptionkey@0"
I20260927 18:47:59.825151 4981 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:47:59.825860 4996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:47:59.826035 4981 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260927 18:47:59.826131 4981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "4120f24d88e84db78416300c24311c23"
format_stamp: "Formatted at 2026-09-27 18:47:59 on dist-test-slave-vs05"
server_key: "0c01e627cfa2d58d1abdcf1dc5580202"
server_key_iv: "a1e5bf2ab53fbd957b866e4b451ffb41"
server_key_version: "encryptionkey@0"
I20260927 18:47:59.826215 4981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:47:59.850704 4981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:47:59.853140 4981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:47:59.853307 4981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:47:59.857484 4981 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:43197
I20260927 18:47:59.857537 5048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:43197 every 8 connection(s)
I20260927 18:47:59.857923 4981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:47:59.858800 5049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:47:59.861151 5049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: Bootstrap starting.
I20260927 18:47:59.861686 5049 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: Neither blocks nor log segments found. Creating new log.
I20260927 18:47:59.861923 5049 log.cc:824] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: Log is configured to *not* fsync() on all Append() calls
I20260927 18:47:59.862596 5049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: No bootstrap required, opened a new log
I20260927 18:47:59.863384 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 4981
I20260927 18:47:59.863504 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:47:59.863693 15803 external_mini_cluster.cc:1428] Setting key 262bcc0de588ffa73097e537ef722828
I20260927 18:47:59.864886 5049 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } }
I20260927 18:47:59.865065 5049 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:47:59.865103 5049 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4120f24d88e84db78416300c24311c23, State: Initialized, Role: FOLLOWER
I20260927 18:47:59.865247 5049 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } }
I20260927 18:47:59.865338 5049 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:47:59.865370 5049 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:47:59.865410 5049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:47:59.866144 5049 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } }
I20260927 18:47:59.866266 5049 leader_election.cc:302] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4120f24d88e84db78416300c24311c23; no voters:
I20260927 18:47:59.866416 5049 leader_election.cc:288] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:47:59.866463 5054 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:47:59.866608 5054 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [term 1 LEADER]: Becoming Leader. State: Replica: 4120f24d88e84db78416300c24311c23, State: Running, Role: LEADER
I20260927 18:47:59.866657 5049 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:47:59.866693 5054 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } }
I20260927 18:47:59.867044 5054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4120f24d88e84db78416300c24311c23. Latest consensus state: current_term: 1 leader_uuid: "4120f24d88e84db78416300c24311c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } } }
I20260927 18:47:59.867141 5054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [sys.catalog]: This master's current role is: LEADER
I20260927 18:47:59.867054 5055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4120f24d88e84db78416300c24311c23" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4120f24d88e84db78416300c24311c23" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 43197 } } }
I20260927 18:47:59.867215 5055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23 [sys.catalog]: This master's current role is: LEADER
W20260927 18:47:59.868443 5070 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:47:59.869480 5070 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:47:59.869560 5062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:47:59.869755 5062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:47:59.870388 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:59.864245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44021 (local address 127.15.110.254:43197)
0927 18:47:59.864461 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:59.864472 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:59.864508 (+ 36us) server_negotiation.cc:407] Connection header received
0927 18:47:59.864847 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:59.864856 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:59.864923 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:59.865081 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:59.865348 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:59.867164 (+ 1816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:59.868382 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:59.868796 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:59.868918 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:59.869655 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:59.869660 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:59.869661 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:59.869878 (+ 217us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:59.869915 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:59.869930 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:59.870016 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:59.870134 (+ 118us) server_negotiation.cc:299] Negotiation successful
0927 18:47:59.870162 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":81,"threads_started":1}
I20260927 18:47:59.871449 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:47:59.872727 5062 catalog_manager.cc:1347] Generated new cluster ID: a7a58b587eea41578e15fc08ce2d867a
I20260927 18:47:59.872783 5062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:47:59.878046 5053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:47:59.875271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36525 (local address 127.15.110.254:43197)
0927 18:47:59.875310 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:47:59.875312 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:47:59.875321 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:47:59.875342 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:47:59.875342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:47:59.875352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:47:59.875435 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:47:59.875585 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:59.876066 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:59.876788 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:47:59.877027 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:47:59.877114 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:47:59.877799 (+ 685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:47:59.877800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:47:59.877801 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:47:59.877805 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:47:59.877819 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:47:59.877824 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:47:59.877877 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:47:59.877937 (+ 60us) server_negotiation.cc:299] Negotiation successful
0927 18:47:59.877949 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:47:59.882404 5062 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:47:59.883098 5062 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:47:59.890894 5062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4120f24d88e84db78416300c24311c23: Generated new TSK 0
I20260927 18:47:59.891113 5062 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:47:59.897274 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 4981
W20260927 18:47:59.901579 4966 connection.cc:535] client connection to 127.15.110.254:43197 recv error: Network error: failed to read from TLS socket (remote: 127.15.110.254:43197): Connection reset by peer (error 104)
2026-09-27T18:47:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1320 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:47:59 dist-test-slave-vs05 krb5kdc[5080](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:47:59 dist-test-slave-vs05 krb5kdc[5080](info): set up 2 sockets
Sep 27 18:47:59 dist-test-slave-vs05 krb5kdc[5080](info): commencing operation
krb5kdc: starting...
W20260927 18:48:00.927516 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.012s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:01 dist-test-slave-vs05 krb5kdc[5080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:01.049841 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41185
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:41185
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:01.132524 5093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:01.132733 5093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:01.132758 5093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:01.133931 5093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:01.133975 5093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:01.133988 5093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:01.134001 5093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:01.134012 5093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:01.135541 5093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:41185
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:41185
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:01.135788 5093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:01.136022 5093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:01.140820 5098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:01.140823 5099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:01.141172 5093 server_base.cc:1034] running on GCE node
W20260927 18:48:01.140820 5101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:01.141530 5093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:01.141851 5093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:01.143014 5093 hybrid_clock.cc:648] HybridClock initialized: now 1790534881143016 us; error 50 us; skew 500 ppm
Sep 27 18:48:01 dist-test-slave-vs05 krb5kdc[5080](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534881, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:01.145838 5093 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:01.146309 5093 webserver.cc:460] Webserver started at http://127.15.110.254:43401/ using document root <none> and password file <none>
I20260927 18:48:01.146587 5093 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:01.146665 5093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:01.146835 5093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:01.148061 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "43ea1a5c556443579cc2aa1828880987"
format_stamp: "Formatted at 2026-09-27 18:48:01 on dist-test-slave-vs05"
server_key: "a68503cb399e9e107686f1d12613c8e5"
server_key_iv: "c78bde1ceec1feafaf7cb2dc22e55eda"
server_key_version: "encryptionkey@0"
I20260927 18:48:01.148463 5093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "43ea1a5c556443579cc2aa1828880987"
format_stamp: "Formatted at 2026-09-27 18:48:01 on dist-test-slave-vs05"
server_key: "a68503cb399e9e107686f1d12613c8e5"
server_key_iv: "c78bde1ceec1feafaf7cb2dc22e55eda"
server_key_version: "encryptionkey@0"
I20260927 18:48:01.149896 5093 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:48:01.150702 5108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:01.150893 5093 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:01.150995 5093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "43ea1a5c556443579cc2aa1828880987"
format_stamp: "Formatted at 2026-09-27 18:48:01 on dist-test-slave-vs05"
server_key: "a68503cb399e9e107686f1d12613c8e5"
server_key_iv: "c78bde1ceec1feafaf7cb2dc22e55eda"
server_key_version: "encryptionkey@0"
I20260927 18:48:01.151084 5093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:01.154699 5093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:01.157274 5093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:01.157465 5093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:01.161803 5093 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:41185
I20260927 18:48:01.161855 5160 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:41185 every 8 connection(s)
I20260927 18:48:01.162212 5093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:01.163383 5161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:01.164620 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5093
I20260927 18:48:01.164781 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:01.164958 15803 external_mini_cluster.cc:1428] Setting key 8caf29e113b4b43a5cacdbfb0c39e2cf
I20260927 18:48:01.166096 5161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: Bootstrap starting.
I20260927 18:48:01.166795 5161 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:01.167127 5161 log.cc:824] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:01.167768 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:01.165549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47215 (local address 127.15.110.254:41185)
0927 18:48:01.165744 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:01.165754 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:01.165791 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:48:01.166288 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:01.166296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:01.166322 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:01.166445 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:01.167235 (+ 790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:01.167245 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:01.167245 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:01.167265 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:01.167310 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:01.167330 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:01.167428 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:01.167490 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:48:01.167533 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":93,"threads_started":1}
I20260927 18:48:01.168004 5161 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: No bootstrap required, opened a new log
I20260927 18:48:01.168830 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:48:01.170010 5161 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } }
I20260927 18:48:01.170192 5161 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:48:01.170233 5161 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43ea1a5c556443579cc2aa1828880987, State: Initialized, Role: FOLLOWER
I20260927 18:48:01.170405 5161 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } }
I20260927 18:48:01.170517 5161 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:48:01.170563 5161 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:48:01.170614 5161 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:01.171149 5161 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } }
I20260927 18:48:01.171303 5161 leader_election.cc:302] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43ea1a5c556443579cc2aa1828880987; no voters:
I20260927 18:48:01.171469 5161 leader_election.cc:288] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:48:01.171525 5171 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:01.171708 5171 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [term 1 LEADER]: Becoming Leader. State: Replica: 43ea1a5c556443579cc2aa1828880987, State: Running, Role: LEADER
I20260927 18:48:01.171770 5161 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:01.171857 5171 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } }
I20260927 18:48:01.172029 5164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:01.170217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48791 (local address 127.15.110.254:41185)
0927 18:48:01.170293 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:01.170295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:01.170309 (+ 14us) server_negotiation.cc:407] Connection header received
0927 18:48:01.170375 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:01.170376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:01.170380 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:01.170446 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:01.171680 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:01.171681 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:01.171682 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:01.171686 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:01.171700 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:01.171704 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:01.171750 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:01.171938 (+ 188us) server_negotiation.cc:299] Negotiation successful
0927 18:48:01.171954 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260927 18:48:01.172646 5171 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43ea1a5c556443579cc2aa1828880987. Latest consensus state: current_term: 1 leader_uuid: "43ea1a5c556443579cc2aa1828880987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } } }
I20260927 18:48:01.172812 5171 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:01.172892 5172 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43ea1a5c556443579cc2aa1828880987" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43ea1a5c556443579cc2aa1828880987" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 41185 } } }
I20260927 18:48:01.173014 5172 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987 [sys.catalog]: This master's current role is: LEADER
W20260927 18:48:01.173602 5186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:01.173647 5186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:01.173712 5185 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:01.173851 5185 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:01.175395 5185 catalog_manager.cc:1347] Generated new cluster ID: 16463237b8594a2ab46e0f514540118a
I20260927 18:48:01.175437 5185 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:48:01.192281 5185 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:01.192499 5185 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:01.199348 5185 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43ea1a5c556443579cc2aa1828880987: Generated new TSK 0
I20260927 18:48:01.199555 5185 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:48:01.208391 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5093
2026-09-27T18:48:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1310 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:48:01 dist-test-slave-vs05 krb5kdc[5192](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:48:01 dist-test-slave-vs05 krb5kdc[5192](info): set up 2 sockets
Sep 27 18:48:01 dist-test-slave-vs05 krb5kdc[5192](info): commencing operation
krb5kdc: starting...
W20260927 18:48:02.251328 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:02.369814 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33243
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:41221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:33243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:02.451028 5205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:02.451319 5205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:02.451349 5205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:02.452507 5205 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:02.452548 5205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:02.452562 5205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:02.452575 5205 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:02.452587 5205 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:02.454061 5205 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:41221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:33243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:33243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:02.454314 5205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:02.454535 5205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:02.459613 5213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:02.459612 5210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:02.459657 5211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:02.459758 5205 server_base.cc:1034] running on GCE node
I20260927 18:48:02.460199 5205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:02.460492 5205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:02.461658 5205 hybrid_clock.cc:648] HybridClock initialized: now 1790534882461653 us; error 41 us; skew 500 ppm
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5192](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534882, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:02.464103 5205 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:02.464586 5205 webserver.cc:460] Webserver started at http://127.15.110.254:40713/ using document root <none> and password file <none>
I20260927 18:48:02.464808 5205 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:02.464879 5205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:02.465029 5205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:02.466068 5205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "f1b7a1fd6045404a9a079f7bea1179a6"
format_stamp: "Formatted at 2026-09-27 18:48:02 on dist-test-slave-vs05"
server_key: "541badad8125c56ed8f873c89b287c6d"
server_key_iv: "99f1faae8fbd6aa38351ee86f030a094"
server_key_version: "encryptionkey@0"
I20260927 18:48:02.466435 5205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "f1b7a1fd6045404a9a079f7bea1179a6"
format_stamp: "Formatted at 2026-09-27 18:48:02 on dist-test-slave-vs05"
server_key: "541badad8125c56ed8f873c89b287c6d"
server_key_iv: "99f1faae8fbd6aa38351ee86f030a094"
server_key_version: "encryptionkey@0"
I20260927 18:48:02.467882 5205 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260927 18:48:02.468627 5220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:02.468850 5205 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:02.468951 5205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "f1b7a1fd6045404a9a079f7bea1179a6"
format_stamp: "Formatted at 2026-09-27 18:48:02 on dist-test-slave-vs05"
server_key: "541badad8125c56ed8f873c89b287c6d"
server_key_iv: "99f1faae8fbd6aa38351ee86f030a094"
server_key_version: "encryptionkey@0"
I20260927 18:48:02.469033 5205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:02.492758 5205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:02.494824 5205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:02.494980 5205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:02.499029 5205 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:33243
I20260927 18:48:02.499066 5272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:33243 every 8 connection(s)
I20260927 18:48:02.499475 5205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:02.500360 5273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:02.502635 5273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6: Bootstrap starting.
I20260927 18:48:02.503257 5273 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:02.503543 5273 log.cc:824] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:02.504223 5273 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6: No bootstrap required, opened a new log
I20260927 18:48:02.504654 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5205
I20260927 18:48:02.504791 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:02.504998 15803 external_mini_cluster.cc:1428] Setting key 7e318787ab0fef44f2d259e2b1025647
I20260927 18:48:02.506500 5273 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } }
I20260927 18:48:02.506677 5273 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:48:02.506714 5273 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1b7a1fd6045404a9a079f7bea1179a6, State: Initialized, Role: FOLLOWER
I20260927 18:48:02.506880 5273 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } }
I20260927 18:48:02.506946 5273 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:48:02.506968 5273 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:48:02.507035 5273 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:02.508078 5273 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } }
I20260927 18:48:02.508183 5273 leader_election.cc:302] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1b7a1fd6045404a9a079f7bea1179a6; no voters:
I20260927 18:48:02.508322 5273 leader_election.cc:288] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:48:02.508383 5278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:02.508561 5278 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [term 1 LEADER]: Becoming Leader. State: Replica: f1b7a1fd6045404a9a079f7bea1179a6, State: Running, Role: LEADER
I20260927 18:48:02.508564 5273 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:02.508698 5278 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } }
I20260927 18:48:02.509030 5278 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1b7a1fd6045404a9a079f7bea1179a6. Latest consensus state: current_term: 1 leader_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } } }
I20260927 18:48:02.509161 5278 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:02.509605 5286 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:02.510131 5279 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1b7a1fd6045404a9a079f7bea1179a6" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 33243 } } }
I20260927 18:48:02.510231 5279 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:02.510121 5286 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:02.512194 5286 catalog_manager.cc:1347] Generated new cluster ID: 18954ca29e134764ac618a701520b0c3
I20260927 18:48:02.512255 5286 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5192](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:02.517711 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:02.505552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43691 (local address 127.15.110.254:33243)
0927 18:48:02.505751 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:02.505762 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:02.505799 (+ 37us) server_negotiation.cc:407] Connection header received
0927 18:48:02.506460 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:02.506469 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:02.506535 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:02.506689 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:02.507542 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:02.508960 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:02.510150 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:02.510790 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:02.510897 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:02.513686 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:02.513690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:02.513690 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:02.513702 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:02.515798 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:02.516201 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:02.516205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:02.516205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:02.516355 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:02.516577 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:02.516578 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:02.516578 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:02.516918 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:02.517106 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:02.517194 (+ 88us) server_negotiation.cc:299] Negotiation successful
0927 18:48:02.517493 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":88,"threads_started":1}
I20260927 18:48:02.518712 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:48:02.525511 5286 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:02.526291 5286 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:02.527568 5277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:02.520169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53333 (local address 127.15.110.254:33243)
0927 18:48:02.520244 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:02.520256 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:02.520371 (+ 115us) server_negotiation.cc:407] Connection header received
0927 18:48:02.520429 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:02.520430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:02.520439 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:02.520494 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:02.521109 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:02.521829 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:02.522868 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:02.523198 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:02.523263 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:02.524700 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:02.524702 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:02.524703 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:02.524710 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:02.526359 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:02.526748 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:02.526749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:02.526749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:02.526814 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:02.527018 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:02.527019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:02.527019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:02.527151 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:02.527303 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:02.527382 (+ 79us) server_negotiation.cc:299] Negotiation successful
0927 18:48:02.527476 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260927 18:48:02.532538 5286 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1b7a1fd6045404a9a079f7bea1179a6: Generated new TSK 0
I20260927 18:48:02.532752 5286 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:48:02.542193 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5205
2026-09-27T18:48:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5304](info): set up 2 sockets
Sep 27 18:48:02 dist-test-slave-vs05 krb5kdc[5304](info): commencing operation
krb5kdc: starting...
W20260927 18:48:03.579416 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:03 dist-test-slave-vs05 krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:03.701767 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40013
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:33809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:40013
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:03.781320 5317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:03.781529 5317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:03.781553 5317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:03.782637 5317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:03.782692 5317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:03.782709 5317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:03.782722 5317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:03.782734 5317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:03.784142 5317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:33809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:40013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:40013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:03.784334 5317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:03.784497 5317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:03.789255 5323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:03.789264 5322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:03.789264 5325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:03.789448 5317 server_base.cc:1034] running on GCE node
I20260927 18:48:03.789667 5317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:03.789980 5317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:03.791157 5317 hybrid_clock.cc:648] HybridClock initialized: now 1790534883791151 us; error 37 us; skew 500 ppm
Sep 27 18:48:03 dist-test-slave-vs05 krb5kdc[5304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534883, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:03.800829 5317 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:03.801347 5317 webserver.cc:460] Webserver started at http://127.15.110.254:42409/ using document root <none> and password file <none>
I20260927 18:48:03.801568 5317 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:03.801635 5317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:03.801782 5317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:03.802876 5317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "7a75733df61f4afca2bc43dec121380b"
format_stamp: "Formatted at 2026-09-27 18:48:03 on dist-test-slave-vs05"
server_key: "1e6661773bfdb5a587ad03044ba83006"
server_key_iv: "0c295ac5614e9820654a4ad1851194e1"
server_key_version: "encryptionkey@0"
I20260927 18:48:03.803300 5317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "7a75733df61f4afca2bc43dec121380b"
format_stamp: "Formatted at 2026-09-27 18:48:03 on dist-test-slave-vs05"
server_key: "1e6661773bfdb5a587ad03044ba83006"
server_key_iv: "0c295ac5614e9820654a4ad1851194e1"
server_key_version: "encryptionkey@0"
I20260927 18:48:03.804658 5317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:03.805454 5332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:03.805646 5317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:03.805745 5317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "7a75733df61f4afca2bc43dec121380b"
format_stamp: "Formatted at 2026-09-27 18:48:03 on dist-test-slave-vs05"
server_key: "1e6661773bfdb5a587ad03044ba83006"
server_key_iv: "0c295ac5614e9820654a4ad1851194e1"
server_key_version: "encryptionkey@0"
I20260927 18:48:03.805830 5317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:03.817917 5317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:03.820124 5317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:03.820290 5317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:03.824316 5317 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:40013
I20260927 18:48:03.824353 5384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:40013 every 8 connection(s)
I20260927 18:48:03.824700 5317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:03.825521 5385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:03.826599 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5317
I20260927 18:48:03.826735 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:03.826908 15803 external_mini_cluster.cc:1428] Setting key 344c4b5d11d79f8fad87292e61821a2c
I20260927 18:48:03.829135 5385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: Bootstrap starting.
I20260927 18:48:03.829840 5385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:03.830145 5385 log.cc:824] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:03.830909 5385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: No bootstrap required, opened a new log
I20260927 18:48:03.833169 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:03.827426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50655 (local address 127.15.110.254:40013)
0927 18:48:03.827617 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:03.827627 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:03.827659 (+ 32us) server_negotiation.cc:407] Connection header received
0927 18:48:03.828212 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:03.828221 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:03.828288 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:03.828453 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:03.828679 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:03.830035 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:03.830903 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:03.831554 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:03.831670 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:03.832419 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:03.832425 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:03.832425 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:03.832657 (+ 232us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:03.832704 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:03.832723 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:03.832804 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:03.832873 (+ 69us) server_negotiation.cc:299] Negotiation successful
0927 18:48:03.832956 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260927 18:48:03.833263 5385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } }
I20260927 18:48:03.833429 5385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:48:03.833459 5385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a75733df61f4afca2bc43dec121380b, State: Initialized, Role: FOLLOWER
I20260927 18:48:03.833628 5385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } }
I20260927 18:48:03.833691 5385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:48:03.833734 5385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:48:03.833779 5385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:03.834316 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:48:03.834504 5385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } }
I20260927 18:48:03.834617 5385 leader_election.cc:302] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a75733df61f4afca2bc43dec121380b; no voters:
I20260927 18:48:03.834791 5385 leader_election.cc:288] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:48:03.834842 5390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:03.835006 5390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [term 1 LEADER]: Becoming Leader. State: Replica: 7a75733df61f4afca2bc43dec121380b, State: Running, Role: LEADER
I20260927 18:48:03.835060 5385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:03.835733 5390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } }
I20260927 18:48:03.836391 5392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a75733df61f4afca2bc43dec121380b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } } }
I20260927 18:48:03.836501 5392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:03.836779 5391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a75733df61f4afca2bc43dec121380b. Latest consensus state: current_term: 1 leader_uuid: "7a75733df61f4afca2bc43dec121380b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a75733df61f4afca2bc43dec121380b" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 40013 } } }
I20260927 18:48:03.837157 5391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b [sys.catalog]: This master's current role is: LEADER
W20260927 18:48:03.837436 5409 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:03.837497 5409 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:03.837589 5411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:03.837765 5411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:03.839624 5411 catalog_manager.cc:1347] Generated new cluster ID: 469c26f578784353996e9c3f08b131a2
I20260927 18:48:03.839676 5411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:48:03.839931 5388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:03.836266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35571 (local address 127.15.110.254:40013)
0927 18:48:03.836304 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:03.836305 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:03.836507 (+ 202us) server_negotiation.cc:407] Connection header received
0927 18:48:03.836596 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:03.836597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:03.836609 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:03.836671 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:03.836898 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:03.837595 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:03.838623 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:03.838922 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:03.839005 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:03.839685 (+ 680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:03.839686 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:03.839686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:03.839690 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:03.839706 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:03.839710 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:03.839769 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:03.839836 (+ 67us) server_negotiation.cc:299] Negotiation successful
0927 18:48:03.839860 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:48:03.842608 5411 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:03.843235 5411 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:03.845646 5411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a75733df61f4afca2bc43dec121380b: Generated new TSK 0
I20260927 18:48:03.845824 5411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:48:03.851050 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5317
2026-09-27T18:48:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1308 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:48:03 dist-test-slave-vs05 krb5kdc[5416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:48:03 dist-test-slave-vs05 krb5kdc[5416](info): set up 2 sockets
Sep 27 18:48:03 dist-test-slave-vs05 krb5kdc[5416](info): commencing operation
krb5kdc: starting...
W20260927 18:48:04.903463 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:04 dist-test-slave-vs05 krb5kdc[5416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534884, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:05.026536 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36479
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:38249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36479
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:05.112087 5429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:05.112299 5429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:05.112329 5429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:05.114070 5429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:05.114128 5429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:05.114149 5429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:05.114172 5429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:05.114193 5429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:05.116019 5429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:38249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36479
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36479
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:05.116219 5429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:05.116389 5429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:05.121145 5435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:05.121153 5434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:05.121165 5437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:05.121336 5429 server_base.cc:1034] running on GCE node
I20260927 18:48:05.121570 5429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:05.121886 5429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:05.123063 5429 hybrid_clock.cc:648] HybridClock initialized: now 1790534885123044 us; error 36 us; skew 500 ppm
Sep 27 18:48:05 dist-test-slave-vs05 krb5kdc[5416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534885, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:05.125455 5429 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:05.125833 5429 webserver.cc:460] Webserver started at http://127.15.110.254:43635/ using document root <none> and password file <none>
I20260927 18:48:05.126055 5429 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:05.126099 5429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:05.126245 5429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:05.127365 5429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "90780cfce41345259aeb3f0b1f9c7054"
format_stamp: "Formatted at 2026-09-27 18:48:05 on dist-test-slave-vs05"
server_key: "60bb7d40561cdabbe95c3135d0512cf8"
server_key_iv: "2b99ada273daa1a0bbcad4837a4105b9"
server_key_version: "encryptionkey@0"
I20260927 18:48:05.127715 5429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "90780cfce41345259aeb3f0b1f9c7054"
format_stamp: "Formatted at 2026-09-27 18:48:05 on dist-test-slave-vs05"
server_key: "60bb7d40561cdabbe95c3135d0512cf8"
server_key_iv: "2b99ada273daa1a0bbcad4837a4105b9"
server_key_version: "encryptionkey@0"
I20260927 18:48:05.129019 5429 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260927 18:48:05.129738 5444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:05.129916 5429 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:48:05.130020 5429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "90780cfce41345259aeb3f0b1f9c7054"
format_stamp: "Formatted at 2026-09-27 18:48:05 on dist-test-slave-vs05"
server_key: "60bb7d40561cdabbe95c3135d0512cf8"
server_key_iv: "2b99ada273daa1a0bbcad4837a4105b9"
server_key_version: "encryptionkey@0"
I20260927 18:48:05.130096 5429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:05.133641 5429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:05.136500 5429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:05.136663 5429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:05.140786 5429 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:36479
I20260927 18:48:05.140834 5496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:36479 every 8 connection(s)
I20260927 18:48:05.141142 5429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:05.141765 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5429
I20260927 18:48:05.141912 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:05.142096 15803 external_mini_cluster.cc:1428] Setting key 4a91576a7c36f091c3761b1ffa7b06d2
I20260927 18:48:05.142247 5497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:05.144795 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:05.142624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53691 (local address 127.15.110.254:36479)
0927 18:48:05.142868 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:05.142877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:05.142907 (+ 30us) server_negotiation.cc:407] Connection header received
0927 18:48:05.143527 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:05.143537 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:05.143579 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:05.143725 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:05.144381 (+ 656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:05.144385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:05.144385 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:05.144389 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:05.144433 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:05.144446 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:05.144520 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:05.144582 (+ 62us) server_negotiation.cc:299] Negotiation successful
0927 18:48:05.144622 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":82,"threads_started":1}
I20260927 18:48:05.145800 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:48:05.145891 5497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054: Bootstrap starting.
I20260927 18:48:05.146586 5497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:05.146937 5497 log.cc:824] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:05.147708 5497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054: No bootstrap required, opened a new log
I20260927 18:48:05.149422 5497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } }
I20260927 18:48:05.149551 5497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260927 18:48:05.149574 5497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90780cfce41345259aeb3f0b1f9c7054, State: Initialized, Role: FOLLOWER
I20260927 18:48:05.149722 5497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } }
I20260927 18:48:05.149794 5497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260927 18:48:05.149816 5497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260927 18:48:05.149883 5497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:05.150409 5497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } }
I20260927 18:48:05.150518 5497 leader_election.cc:302] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 90780cfce41345259aeb3f0b1f9c7054; no voters:
I20260927 18:48:05.150687 5497 leader_election.cc:288] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260927 18:48:05.150758 5507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:05.150866 5500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:05.147379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43385 (local address 127.15.110.254:36479)
0927 18:48:05.149760 (+ 2381us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:05.149762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:05.149770 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:05.149802 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:05.149803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:05.149809 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:05.149901 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:05.150539 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:05.150541 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:05.150541 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0927 18:48:05.150545 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:05.150558 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0927 18:48:05.150563 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:05.150626 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:05.150739 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 18:48:05.150753 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2365}
I20260927 18:48:05.151022 5497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:05.151057 5507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [term 1 LEADER]: Becoming Leader. State: Replica: 90780cfce41345259aeb3f0b1f9c7054, State: Running, Role: LEADER
I20260927 18:48:05.151254 5507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } }
I20260927 18:48:05.151672 5509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 90780cfce41345259aeb3f0b1f9c7054. Latest consensus state: current_term: 1 leader_uuid: "90780cfce41345259aeb3f0b1f9c7054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } } }
I20260927 18:48:05.151759 5509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:05.151968 5516 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:05.151973 5508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "90780cfce41345259aeb3f0b1f9c7054" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "90780cfce41345259aeb3f0b1f9c7054" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36479 } } }
I20260927 18:48:05.152321 5508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:05.152488 5516 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:05.154012 5516 catalog_manager.cc:1347] Generated new cluster ID: 079c85552e134fd0b20d874097f2e036
I20260927 18:48:05.154062 5516 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:48:05.164980 5516 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:05.165153 5516 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:05.167575 5516 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 90780cfce41345259aeb3f0b1f9c7054: Generated new TSK 0
I20260927 18:48:05.167750 5516 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260927 18:48:05.170945 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5429
2026-09-27T18:48:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1323 ms)
[----------] 18 tests from AuthTokenIssuingTest (23798 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:48:05 dist-test-slave-vs05 krb5kdc[5528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:48:05 dist-test-slave-vs05 krb5kdc[5528](info): set up 2 sockets
Sep 27 18:48:05 dist-test-slave-vs05 krb5kdc[5528](info): commencing operation
krb5kdc: starting...
W20260927 18:48:06.207450 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.014s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:06.326789 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46861
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:06.409546 5541 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:06.409798 5541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:06.409829 5541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:06.409874 5541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:06.411033 5541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:06.411104 5541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:06.411129 5541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:06.411157 5541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:06.411222 5541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:06.412747 5541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:46861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:06.413040 5541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:06.413304 5541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:06.418375 5549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.418403 5546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.418412 5547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:06.418568 5541 server_base.cc:1034] running on GCE node
I20260927 18:48:06.418885 5541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:06.419134 5541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:06.420354 5541 hybrid_clock.cc:648] HybridClock initialized: now 1790534886420334 us; error 41 us; skew 500 ppm
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:06.422993 5541 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:06.423447 5541 webserver.cc:460] Webserver started at http://127.15.110.254:36071/ using document root <none> and password file <none>
I20260927 18:48:06.423682 5541 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:06.423755 5541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:06.423909 5541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:06.424998 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "6c6d7bf49c824976a18245a2e4d74ded"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "e61a68331e67fe731bf3da1f8b168833"
server_key_iv: "7e323baae9556b9d8200aeea1ddba203"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.425370 5541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "6c6d7bf49c824976a18245a2e4d74ded"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "e61a68331e67fe731bf3da1f8b168833"
server_key_iv: "7e323baae9556b9d8200aeea1ddba203"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.426776 5541 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260927 18:48:06.427609 5556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:06.427783 5541 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:06.427896 5541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "6c6d7bf49c824976a18245a2e4d74ded"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "e61a68331e67fe731bf3da1f8b168833"
server_key_iv: "7e323baae9556b9d8200aeea1ddba203"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.427989 5541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:06.445709 5541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:06.448181 5541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:06.448357 5541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:06.452466 5541 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:46861
I20260927 18:48:06.452502 5608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:46861 every 8 connection(s)
I20260927 18:48:06.452865 5541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:06.453706 5609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:06.455281 5609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:06.461583 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5541
I20260927 18:48:06.461714 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:06.461902 15803 external_mini_cluster.cc:1428] Setting key cc304219344dd45931d9f035a13ca219
I20260927 18:48:06.465263 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.456003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52360 (local address 127.15.110.254:46861)
0927 18:48:06.456155 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.456163 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.456261 (+ 98us) server_negotiation.cc:407] Connection header received
0927 18:48:06.456807 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.456809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.456825 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.456903 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.457410 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.458217 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.459101 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.459492 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.459573 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.461535 (+ 1962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.461545 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.461545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.461554 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.463665 (+ 2111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.464142 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.464145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.464145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.464224 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.464480 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.464480 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.464480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.464679 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.464736 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.464981 (+ 245us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.465102 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":127,"thread_start_us":104,"threads_started":1}
I20260927 18:48:06.465380 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.455955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46861 (local address 127.0.0.1:52360)
0927 18:48:06.456114 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.456156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.456670 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.456936 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.456938 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.457172 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.457326 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.457329 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.458312 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.458313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.459021 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.459023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.459564 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.459565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.459620 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.460061 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.460069 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.461449 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.463753 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.463760 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.463762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.464057 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.464291 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.464292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.464292 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.464402 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.464747 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.464753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.464837 (+ 84us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.464940 (+ 103us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.465193 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":91,"thread_start_us":60,"threads_started":1}
I20260927 18:48:06.466171 5609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:06.466555 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.466425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33394)
0927 18:48:06.466476 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.466530 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:48:06.466718 5558 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
I20260927 18:48:06.467216 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.467081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33406)
0927 18:48:06.467133 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.467167 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260927 18:48:06.467334 5609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35871: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.110.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
I20260927 18:48:06.487711 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:35871
--webserver_interface=127.15.110.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:48:06.516593 5609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } attempt: 1
I20260927 18:48:06.517141 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.516972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33410)
0927 18:48:06.517049 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.517101 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260927 18:48:06.517535 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.517431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33416)
0927 18:48:06.517480 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.517512 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260927 18:48:06.517665 5609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35871: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
W20260927 18:48:06.568363 5616 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:06.568567 5616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:06.568584 5616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:06.568641 5616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:06.569793 5616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:06.569833 5616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:06.569847 5616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:06.569901 5616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:06.569919 5616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:06.571395 5616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.110.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:06.571640 5616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:06.571861 5616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:06.577234 5621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.577265 5624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.577266 5622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:06.577430 5616 server_base.cc:1034] running on GCE node
I20260927 18:48:06.577785 5616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:06.578073 5616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:06.579275 5616 hybrid_clock.cc:648] HybridClock initialized: now 1790534886579267 us; error 36 us; skew 500 ppm
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:06.581673 5616 init.cc:377] Logged in from keytab as kudu/127.15.110.253@KRBTEST.COM (short username kudu)
I20260927 18:48:06.582140 5616 webserver.cc:460] Webserver started at http://127.15.110.253:37973/ using document root <none> and password file <none>
I20260927 18:48:06.582371 5616 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:06.582439 5616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:06.582588 5616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:06.583772 5616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data/instance:
uuid: "977d13757d144bcb89f3454fc572b56f"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "6db5f48bd3913b8df42ea7a6f9e66548"
server_key_iv: "60e81b53160397681d6eb518660644c6"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.584126 5616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal/instance:
uuid: "977d13757d144bcb89f3454fc572b56f"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "6db5f48bd3913b8df42ea7a6f9e66548"
server_key_iv: "60e81b53160397681d6eb518660644c6"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.585402 5616 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:48:06.585827 5609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } attempt: 2
I20260927 18:48:06.586191 5631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:06.586239 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.586087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33424)
0927 18:48:06.586174 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.586210 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 18:48:06.586440 5616 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:06.586542 5616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "977d13757d144bcb89f3454fc572b56f"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "6db5f48bd3913b8df42ea7a6f9e66548"
server_key_iv: "60e81b53160397681d6eb518660644c6"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.586664 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.586532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33438)
0927 18:48:06.586588 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.586616 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260927 18:48:06.586632 5616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
W20260927 18:48:06.586820 5609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:35871: Network error: Client connection negotiation failed: client connection to 127.15.110.253:35871: connect: Connection refused (error 111)
I20260927 18:48:06.605224 5616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:06.607125 5616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:06.607317 5616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:06.611323 5616 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.253:35871
I20260927 18:48:06.611347 5683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.253:35871 every 8 connection(s)
I20260927 18:48:06.611706 5616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
I20260927 18:48:06.612385 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5616
I20260927 18:48:06.612501 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-1/wal/instance
I20260927 18:48:06.612695 15803 external_mini_cluster.cc:1428] Setting key 479fdea1f9bb11a7de048d8cd3cc4f62
I20260927 18:48:06.612761 5684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:06.615139 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.110.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.252@KRBTEST.COM" created.
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:06.627341 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.616121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52372 (local address 127.15.110.254:46861)
0927 18:48:06.616170 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.616172 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.616396 (+ 224us) server_negotiation.cc:407] Connection header received
0927 18:48:06.617052 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.617053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.617072 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.617148 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.618096 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.618672 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.619797 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.620017 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.620104 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.624356 (+ 4252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.624358 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.624358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.624365 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.625852 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.626247 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.626248 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.626249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.626316 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.626591 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.626593 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.626593 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.626716 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.626796 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.627119 (+ 323us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.627245 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:48:06.627529 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.616090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46861 (local address 127.0.0.1:52372)
0927 18:48:06.616250 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.616292 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.616915 (+ 623us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.617304 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.617308 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.617670 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.617981 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.617990 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.618789 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.618790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.619694 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.619696 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.620114 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.620115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.620247 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.620830 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.620839 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.624233 (+ 3394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.625929 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.625935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.625937 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.626155 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.626387 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.626388 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.626389 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.626519 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.626787 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.626794 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.626972 (+ 178us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.627082 (+ 110us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.627272 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260927 18:48:06.628192 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:06.636682 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.628485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33442)
0927 18:48:06.628560 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.628570 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.628610 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.628874 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.628875 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.629029 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.629093 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.629096 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.629967 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.629968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.630883 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.630886 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.631491 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.631492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.631549 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.631986 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.631991 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.633941 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.635643 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.635644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.635644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.635808 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.636035 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.636036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.636037 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.636094 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.636359 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.636360 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.636396 (+ 36us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.636468 (+ 72us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.636565 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260927 18:48:06.636754 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.628550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33442 (local address 127.15.110.253:35871)
0927 18:48:06.628716 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.628721 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.628739 (+ 18us) server_negotiation.cc:407] Connection header received
0927 18:48:06.628768 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.628770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.628781 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.628844 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.629173 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.629843 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.630966 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.631408 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.631469 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.634047 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.634058 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.634058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.634068 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.635548 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.635902 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.635905 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.635905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.635967 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.636144 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.636145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.636145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.636288 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.636381 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.636498 (+ 117us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.636617 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":136,"thread_start_us":109,"threads_started":1}
I20260927 18:48:06.637504 5684 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:06.637864 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.637741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48658)
0927 18:48:06.637789 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.637838 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:48:06.638005 5632 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
I20260927 18:48:06.638415 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.638320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48668)
0927 18:48:06.638367 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.638392 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260927 18:48:06.638521 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:40675: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
I20260927 18:48:06.639955 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:40675
--webserver_interface=127.15.110.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:43049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:48:06.677980 5609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } attempt: 3
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:06.686182 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.678439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33448)
0927 18:48:06.678518 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.678532 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.678608 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.678758 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.678759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.678922 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.678992 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.678994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.679982 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.679983 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.680523 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.680524 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.681082 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.681083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.681150 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.681644 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.681649 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.683104 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.685112 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.685113 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.685114 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.685293 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.685559 (+ 266us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.685560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.685561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.685619 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.685911 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.685912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.685960 (+ 48us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.686027 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.686108 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260927 18:48:06.686264 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.678541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.15.110.253:35871)
0927 18:48:06.678616 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.678619 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.678632 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:48:06.678665 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.678666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.678680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.678746 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.679068 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.679891 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.680653 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.680995 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.681072 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.683216 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.683217 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.683218 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.683227 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.685020 (+ 1793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.685408 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.685409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.685410 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.685475 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.685684 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.685685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.685685 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.685824 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.685893 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.686070 (+ 177us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.686152 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:48:06.686988 5609 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:06.687381 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.687273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48670)
0927 18:48:06.687324 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.687359 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:48:06.687777 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.687668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48686)
0927 18:48:06.687728 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.687758 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260927 18:48:06.687737 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } attempt: 1
W20260927 18:48:06.687853 5609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:40675: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
I20260927 18:48:06.688050 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.687965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48688)
0927 18:48:06.687999 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.688024 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260927 18:48:06.688359 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.688296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48696)
0927 18:48:06.688326 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.688346 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260927 18:48:06.688457 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:40675: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
I20260927 18:48:06.719022 5609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } attempt: 1
I20260927 18:48:06.719560 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.719409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48704)
0927 18:48:06.719487 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.719530 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260927 18:48:06.720017 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.719922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48714)
0927 18:48:06.719972 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.719999 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260927 18:48:06.720135 5609 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:40675: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
W20260927 18:48:06.725334 5691 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:06.725576 5691 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:06.725615 5691 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:06.725656 5691 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:06.726809 5691 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:06.726873 5691 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:06.726899 5691 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:06.726926 5691 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:06.726954 5691 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:06.728538 5691 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:43049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:46861,127.15.110.253:35871,127.15.110.252:40675
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:40675
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.110.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5691
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:06.728832 5691 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:06.729055 5691 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:06.734333 5696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.734308 5699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:06.734372 5697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:06.734400 5691 server_base.cc:1034] running on GCE node
I20260927 18:48:06.734779 5691 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:06.735077 5691 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:06.736253 5691 hybrid_clock.cc:648] HybridClock initialized: now 1790534886736229 us; error 28 us; skew 500 ppm
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:06.738649 5691 init.cc:377] Logged in from keytab as kudu/127.15.110.252@KRBTEST.COM (short username kudu)
I20260927 18:48:06.739141 5691 webserver.cc:460] Webserver started at http://127.15.110.252:34263/ using document root <none> and password file <none>
I20260927 18:48:06.739408 5691 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:06.739480 5691 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:06.739635 5691 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:06.740734 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data/instance:
uuid: "2c1455720d7c49efb867677448c19a8b"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "89198040881708e7293413e613e37be6"
server_key_iv: "cd5d0ff06656075ef8ece649965a87c4"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.741102 5691 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal/instance:
uuid: "2c1455720d7c49efb867677448c19a8b"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "89198040881708e7293413e613e37be6"
server_key_iv: "cd5d0ff06656075ef8ece649965a87c4"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.742511 5691 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260927 18:48:06.743295 5706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:06.743468 5691 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:48:06.743567 5691 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "2c1455720d7c49efb867677448c19a8b"
format_stamp: "Formatted at 2026-09-27 18:48:06 on dist-test-slave-vs05"
server_key: "89198040881708e7293413e613e37be6"
server_key_iv: "cd5d0ff06656075ef8ece649965a87c4"
server_key_version: "encryptionkey@0"
I20260927 18:48:06.743649 5691 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:06.752487 5691 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:06.754263 5691 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:06.754422 5691 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:06.756570 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } attempt: 2
I20260927 18:48:06.757014 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.756868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48722)
0927 18:48:06.756944 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.756975 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260927 18:48:06.757511 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.757375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48726)
0927 18:48:06.757449 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.757480 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260927 18:48:06.757622 5684 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:40675: Network error: Client connection negotiation failed: client connection to 127.15.110.252:40675: connect: Connection refused (error 111)
I20260927 18:48:06.758836 5691 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.252:40675
I20260927 18:48:06.759164 5758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.252:40675 every 8 connection(s)
I20260927 18:48:06.759367 5691 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
I20260927 18:48:06.760010 5759 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:06.761597 5759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:06.765161 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5691
I20260927 18:48:06.765287 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790534620099873-15803-0/minicluster-data/master-2/wal/instance
I20260927 18:48:06.765480 15803 external_mini_cluster.cc:1428] Setting key a333aa6aa23d22cd031e39cc39c951cc
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:06.771023 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.762334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52378 (local address 127.15.110.254:46861)
0927 18:48:06.762390 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.762392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.762556 (+ 164us) server_negotiation.cc:407] Connection header received
0927 18:48:06.763020 (+ 464us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.763021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.763030 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.763081 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.763736 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.764258 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.765307 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.765603 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.765684 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.768027 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.768028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.768029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.768035 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.769577 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.770059 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.770060 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.770060 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.770110 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.770386 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.770387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.770387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.770485 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.770573 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.770842 (+ 269us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.770926 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:48:06.771056 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.762257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:46861 (local address 127.0.0.1:52378)
0927 18:48:06.762411 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.762465 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.762939 (+ 474us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.763140 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.763144 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.763460 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.763645 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.763650 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.764350 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.764351 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.765176 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.765179 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.765706 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.765707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.765838 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.766402 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.766410 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.767940 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.769655 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.769662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.769664 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.769980 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.770186 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.770187 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.770187 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.770319 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.770586 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.770592 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.770706 (+ 114us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.770792 (+ 86us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.770901 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":86,"threads_started":1}
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:06.771713 5759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:06.775053 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.765905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48740 (local address 127.15.110.252:40675)
0927 18:48:06.766088 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.766100 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.766115 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:48:06.766156 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.766158 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.766175 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.766267 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.766875 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.767707 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.768559 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.768875 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.768958 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.771714 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.771719 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.771720 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.771727 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.773774 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.774131 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.774134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.774135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.774199 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.774392 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.774392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.774393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.774596 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.774740 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.774841 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.774923 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":105,"threads_started":1}
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:06.778883 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.772136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33450 (local address 127.15.110.253:35871)
0927 18:48:06.772205 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.772207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.772216 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:06.772243 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.772244 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.772255 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.772331 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.772716 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.773229 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.774102 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.774376 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.774440 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.776407 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.776408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.776409 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.776414 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.777773 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.778125 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.778126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.778126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.778186 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.778387 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.778387 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.778388 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.778536 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.778611 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.778760 (+ 149us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.778823 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260927 18:48:06.778930 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.772048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:35871 (local address 127.0.0.1:33450)
0927 18:48:06.772104 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.772115 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.772157 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.772358 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.772360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.772553 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.772614 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.772616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.773306 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.773307 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.774025 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.774027 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.774447 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.774448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.774517 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.774936 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.774940 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.776331 (+ 1391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.777852 (+ 1521us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.777853 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.777854 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.778060 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.778259 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.778259 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.778260 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.778320 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.778621 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.778623 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.778664 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.778739 (+ 75us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.778828 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260927 18:48:06.779282 5759 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:06.783531 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.776121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52390 (local address 127.15.110.254:46861)
0927 18:48:06.776179 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.776181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.776189 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:06.776208 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.776209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.776220 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.776271 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.776779 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.777246 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.777957 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.778180 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.778247 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.780801 (+ 2554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.780803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.780803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.780810 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.782331 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.782715 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.782716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.782716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.782783 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.782993 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.782994 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.782994 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.783100 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.783352 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.783395 (+ 43us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.783471 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260927 18:48:06.785888 5761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.779529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48744)
0927 18:48:06.779579 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.779588 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.779636 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.779746 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.779747 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.779918 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.779943 (+ 25us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.779945 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.780615 (+ 670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.780616 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.781154 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.781155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.781528 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.781529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.781577 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.781998 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.782002 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.783104 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.784894 (+ 1790us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.784895 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.784896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.785104 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.785278 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.785278 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.785279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.785338 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.785531 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.785532 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.785573 (+ 41us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.785643 (+ 70us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.785731 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260927 18:48:06.785943 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.779581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48744 (local address 127.15.110.252:40675)
0927 18:48:06.779615 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.779617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.779623 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:48:06.779681 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.779682 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.779692 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.779742 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.780027 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.780534 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.781230 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.781449 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.781519 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.783207 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.783208 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.783208 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.783214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.784821 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.785173 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.785174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.785174 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.785220 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.785392 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.785392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.785392 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.785473 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.785517 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.785659 (+ 142us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.785806 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":20}
I20260927 18:48:06.787743 5759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: Bootstrap starting.
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:06.788715 5759 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:06.789019 5759 log.cc:824] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:06.789889 5759 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: No bootstrap required, opened a new log
I20260927 18:48:06.791841 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.784455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33460 (local address 127.15.110.253:35871)
0927 18:48:06.784493 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.784495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.784503 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:06.784524 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.784525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.784533 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.784600 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.785033 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.785736 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.786627 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.786906 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.786981 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.789093 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.789094 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.789095 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.789102 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.790671 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.791043 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.791044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.791045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.791092 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.791318 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.791318 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.791319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.791436 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.791559 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.791660 (+ 101us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.791736 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:48:06.792075 5759 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.792227 5759 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c1455720d7c49efb867677448c19a8b, State: Initialized, Role: FOLLOWER
I20260927 18:48:06.792351 5759 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.792713 5759 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:06.792871 5765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.792990 5765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.793390 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260927 18:48:06.794662 5776 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:06.794765 5776 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:06.795246 5609 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } attempt: 2
I20260927 18:48:06.794816 5583 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "6c6d7bf49c824976a18245a2e4d74ded"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52390
I20260927 18:48:06.794904 5691 master_runner.cc:186] Error getting master registration for 127.15.110.254:46861: OK, instance_id { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" instance_seqno: 1790534886451286 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:48:06.797235 5691 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:06.802871 5611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.795497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48752)
0927 18:48:06.795607 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.795618 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.795660 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.795896 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.795897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.796100 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.796157 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.796159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.797043 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.797044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.797836 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.797839 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.798325 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.798326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.798388 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.798810 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.798815 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.800103 (+ 1288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.801684 (+ 1581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.801685 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.801686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.801932 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.802150 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.802150 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.802151 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.802220 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.802521 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.802522 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.802574 (+ 52us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.802653 (+ 79us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.802769 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260927 18:48:06.802870 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.795725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.15.110.252:40675)
0927 18:48:06.795759 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.795761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.795767 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:48:06.795786 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.795787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.795797 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.795851 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.796248 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.796942 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.797929 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.798239 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.798310 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.800206 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.800207 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.800208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.800214 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.801614 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.802018 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.802019 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.802019 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.802078 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.802285 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.802286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.802286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.802448 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.802513 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.802695 (+ 182us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.802773 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:06.804847 5609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: Bootstrap starting.
I20260927 18:48:06.805634 5609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:06.805967 5609 log.cc:824] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:06.806922 5609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: No bootstrap required, opened a new log
I20260927 18:48:06.808393 5609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.808559 5609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c6d7bf49c824976a18245a2e4d74ded, State: Initialized, Role: FOLLOWER
I20260927 18:48:06.808637 5609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.808904 5779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.808954 5609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:06.808985 5583 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:48:06.809012 5779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.809114 5583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:06.809983 5583 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.810541 5733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c6d7bf49c824976a18245a2e4d74ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "2c1455720d7c49efb867677448c19a8b"
I20260927 18:48:06.810674 5733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:06.811252 5583 leader_election.cc:288] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [CANDIDATE]: Term 1 election: Requested vote from peers 977d13757d144bcb89f3454fc572b56f (127.15.110.253:35871), 2c1455720d7c49efb867677448c19a8b (127.15.110.252:40675)
I20260927 18:48:06.811601 5733 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6d7bf49c824976a18245a2e4d74ded in term 1.
I20260927 18:48:06.811816 5557 leader_election.cc:302] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c1455720d7c49efb867677448c19a8b, 6c6d7bf49c824976a18245a2e4d74ded; no voters:
I20260927 18:48:06.811949 5779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:06.812095 5779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [term 1 LEADER]: Becoming Leader. State: Replica: 6c6d7bf49c824976a18245a2e4d74ded, State: Running, Role: LEADER
I20260927 18:48:06.812178 5779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.812582 5779 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6d7bf49c824976a18245a2e4d74ded. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.812644 5779 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: This master's current role is: LEADER
W20260927 18:48:06.813513 5790 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:06.813558 5790 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:06.812423 5541 master_runner.cc:186] Error getting master registration for 127.15.110.253:35871: OK, instance_id { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" instance_seqno: 1790534886610133 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:48:06.814553 5541 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:48:06.814806 5798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:06.815022 5798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:06.816875 5733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [term 1 FOLLOWER]: Refusing update from remote peer 6c6d7bf49c824976a18245a2e4d74ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:48:06.817278 5795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:48:06.818992 5765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6d7bf49c824976a18245a2e4d74ded. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.819093 5765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.821476 5795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.821576 5795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:06.822439 5765 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.822532 5765 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.822660 5802 mvcc.cc:205] Tried to move back new op lower bound from 7334030896398811136 to 7334030896383373312. Current Snapshot: MvccSnapshot[applied={T|T < 7334030896398811136}]
I20260927 18:48:06.824849 5803 mvcc.cc:205] Tried to move back new op lower bound from 7334030896398811136 to 7334030896383373312. Current Snapshot: MvccSnapshot[applied={T|T < 7334030896398811136}]
I20260927 18:48:06.825803 5798 catalog_manager.cc:1347] Generated new cluster ID: 6e8ddaaea2d646318271b34f92f71c78
I20260927 18:48:06.825852 5798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:48:06.831636 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.813501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33462 (local address 127.15.110.253:35871)
0927 18:48:06.813544 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.813546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.819317 (+ 5771us) server_negotiation.cc:407] Connection header received
0927 18:48:06.819427 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.819429 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.819439 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.819470 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.820277 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.821068 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.824143 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.824460 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.824506 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.828326 (+ 3820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.828327 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.828328 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.828347 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.829990 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.830570 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.830571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.830571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.830634 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.830939 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.830940 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.830940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.831069 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.831147 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.831468 (+ 321us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.831543 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:48:06.832979 5612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.812809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.15.110.254:46861)
0927 18:48:06.812858 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.812860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.815841 (+ 2981us) server_negotiation.cc:407] Connection header received
0927 18:48:06.815944 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.815945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.815955 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.815984 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.816808 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.817514 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.819050 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.819381 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.819443 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.821365 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.821366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.821366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.821374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.823202 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.824184 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.824185 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.824185 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.824256 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.824614 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.824615 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.824615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.824771 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.825225 (+ 454us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.832692 (+ 7467us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.832774 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:48:06.840065 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.812757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.15.110.252:40675)
0927 18:48:06.812895 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.812896 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.822657 (+ 9761us) server_negotiation.cc:407] Connection header received
0927 18:48:06.822781 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.822782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.822792 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.822821 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.823915 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.824632 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.835283 (+ 10651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.835600 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.835670 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.837218 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.837219 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.837219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.837226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.838908 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.839308 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.839309 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.839309 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.839376 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.839566 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.839567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.839567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.839686 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.839780 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.839899 (+ 119us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.839969 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260927 18:48:06.845072 5798 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:06.845887 5798 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:06.848757 5684 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } attempt: 3
I20260927 18:48:06.853034 5798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded: Generated new TSK 0
I20260927 18:48:06.853543 5798 catalog_manager.cc:1514] Initializing in-progress tserver states...
Sep 27 18:48:06 dist-test-slave-vs05 krb5kdc[5528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534886, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:06.856967 5687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.849027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:40675 (local address 127.0.0.1:48768)
0927 18:48:06.849101 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:06.849111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:06.849144 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:06.849413 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:06.849414 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:06.849585 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:06.849630 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.849631 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.850636 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.850637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.851347 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:06.851356 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.851844 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.851845 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:06.851920 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.852437 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:06.852443 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:06.853983 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:06.855877 (+ 1894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.855878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.855879 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.856090 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.856302 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:06.856303 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:06.856304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:06.856367 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:06.856647 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:06.856648 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:06.856692 (+ 44us) client_negotiation.cc:769] Sending connection context
0927 18:48:06.856760 (+ 68us) client_negotiation.cc:240] Negotiation successful
0927 18:48:06.856869 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260927 18:48:06.856967 5763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.849240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48768 (local address 127.15.110.252:40675)
0927 18:48:06.849286 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.849288 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.849294 (+ 6us) server_negotiation.cc:407] Connection header received
0927 18:48:06.849313 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.849314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.849322 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.849373 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.849697 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.850370 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.851438 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.851722 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.851784 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.854074 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:06.854076 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:06.854077 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:06.854085 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:06.855780 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.856159 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.856161 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.856161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.856213 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:06.856437 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:06.856438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:06.856439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:06.856567 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:06.856624 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.856799 (+ 175us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.856869 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:06.858966 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: Bootstrap starting.
I20260927 18:48:06.860351 5684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:06.860700 5684 log.cc:824] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:06.864163 5684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: No bootstrap required, opened a new log
I20260927 18:48:06.866257 5684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.866401 5684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 977d13757d144bcb89f3454fc572b56f, State: Initialized, Role: FOLLOWER
I20260927 18:48:06.866487 5684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } }
I20260927 18:48:06.866794 5684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:06.866901 5658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "6c6d7bf49c824976a18245a2e4d74ded" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "977d13757d144bcb89f3454fc572b56f"
I20260927 18:48:06.867003 5658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:06.867964 5658 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6c6d7bf49c824976a18245a2e4d74ded in term 1.
I20260927 18:48:06.868371 5657 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [term 1 FOLLOWER]: Refusing update from remote peer 6c6d7bf49c824976a18245a2e4d74ded: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:48:06.868637 5795 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [LEADER]: Connected to new peer: Peer: permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:48:06.871673 5809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.871755 5809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.871800 5809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6c6d7bf49c824976a18245a2e4d74ded. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.871836 5809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.871877 5809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:06.871930 5809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:06.872776 5822 mvcc.cc:205] Tried to move back new op lower bound from 7334030896538865664 to 7334030896383373312. Current Snapshot: MvccSnapshot[applied={T|T < 7334030896398811136}]
I20260927 18:48:06.875905 5820 catalog_manager.cc:1259] Loaded cluster ID: 6e8ddaaea2d646318271b34f92f71c78
I20260927 18:48:06.875954 5820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: loading cluster ID for follower catalog manager: success
I20260927 18:48:06.877195 5820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: acquiring CA information for follower catalog manager: success
I20260927 18:48:06.877508 5820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 977d13757d144bcb89f3454fc572b56f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260927 18:48:06.894280 5689 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.892659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33464 (local address 127.15.110.253:35871)
0927 18:48:06.892703 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.892705 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.893612 (+ 907us) server_negotiation.cc:407] Connection header received
0927 18:48:06.893724 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.893725 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.893737 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.893768 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.894211 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33464: BlockingRecv error: recv got EOF from 127.0.0.1:33464 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 18:48:06.895004 5612 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.893043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.15.110.254:46861)
0927 18:48:06.893085 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.893087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.894507 (+ 1420us) server_negotiation.cc:407] Connection header received
0927 18:48:06.894603 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.894604 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.894615 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.894644 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.894939 (+ 295us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52406: BlockingRecv error: recv got EOF from 127.0.0.1:52406 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260927 18:48:06.895807 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.893353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48780 (local address 127.15.110.252:40675)
0927 18:48:06.893392 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.893394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.895263 (+ 1869us) server_negotiation.cc:407] Connection header received
0927 18:48:06.895356 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.895357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.895368 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.895397 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.895675 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48780: BlockingRecv error: recv got EOF from 127.0.0.1:48780 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:48:06.901041 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.897459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33466 (local address 127.15.110.253:35871)
0927 18:48:06.897495 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.897497 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.897711 (+ 214us) server_negotiation.cc:407] Connection header received
0927 18:48:06.897815 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.897816 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.897855 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.897901 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.898101 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.898889 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.900097 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.900402 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.900445 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.900662 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.900788 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.900839 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.900950 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.900969 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:48:06.904314 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.903559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48794 (local address 127.15.110.252:40675)
0927 18:48:06.903600 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.903602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.903778 (+ 176us) server_negotiation.cc:407] Connection header received
0927 18:48:06.903881 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.903882 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.903899 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.903928 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.904266 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48794: BlockingRecv error: recv got EOF from 127.0.0.1:48794 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:48:06.909626 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.906547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33478 (local address 127.15.110.253:35871)
0927 18:48:06.906581 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.906583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.906804 (+ 221us) server_negotiation.cc:407] Connection header received
0927 18:48:06.906892 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.906893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.906901 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.906927 (+ 26us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.907076 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.907806 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.908857 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.909157 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.909197 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.909389 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.909423 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.909448 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.909540 (+ 92us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.909558 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:48:06.912649 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.911925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48796 (local address 127.15.110.252:40675)
0927 18:48:06.911961 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.911963 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.912137 (+ 174us) server_negotiation.cc:407] Connection header received
0927 18:48:06.912239 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.912240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.912251 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.912280 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.912601 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48796: BlockingRecv error: recv got EOF from 127.0.0.1:48796 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:48:06.919036 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.915803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33494 (local address 127.15.110.253:35871)
0927 18:48:06.915848 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.915850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.916091 (+ 241us) server_negotiation.cc:407] Connection header received
0927 18:48:06.916191 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.916192 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.916201 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.916246 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.916421 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.917115 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.918248 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.918524 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.918569 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.918774 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.918807 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.918836 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.918946 (+ 110us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.918964 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:48:06.922020 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.921300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.15.110.252:40675)
0927 18:48:06.921338 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.921340 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.921515 (+ 175us) server_negotiation.cc:407] Connection header received
0927 18:48:06.921606 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.921607 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.921618 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.921646 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.921970 (+ 324us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48804: BlockingRecv error: recv got EOF from 127.0.0.1:48804 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:06.930366 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.927146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33498 (local address 127.15.110.253:35871)
0927 18:48:06.927207 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.927209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.927456 (+ 247us) server_negotiation.cc:407] Connection header received
0927 18:48:06.927552 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.927554 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.927563 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.927591 (+ 28us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.927764 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.928467 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.929532 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.929805 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.929850 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.930051 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.930084 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.930112 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.930275 (+ 163us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.930294 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260927 18:48:06.933460 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.932603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48806 (local address 127.15.110.252:40675)
0927 18:48:06.932641 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.932642 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.932898 (+ 256us) server_negotiation.cc:407] Connection header received
0927 18:48:06.932998 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.932999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.933011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.933041 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.933406 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48806: BlockingRecv error: recv got EOF from 127.0.0.1:48806 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260927 18:48:06.946110 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.942739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.15.110.253:35871)
0927 18:48:06.942795 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.942797 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.943055 (+ 258us) server_negotiation.cc:407] Connection header received
0927 18:48:06.943156 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.943157 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.943167 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.943243 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.943432 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.944196 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.945314 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.945604 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.945647 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.945848 (+ 201us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.945882 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.945911 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.946017 (+ 106us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.946036 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 18:48:06.949710 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:06.948650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48818 (local address 127.15.110.252:40675)
0927 18:48:06.948692 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.948694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.948879 (+ 185us) server_negotiation.cc:407] Connection header received
0927 18:48:06.948974 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.948975 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.948987 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.949016 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:06.949654 (+ 638us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48818: BlockingRecv error: recv got EOF from 127.0.0.1:48818 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260927 18:48:06.996683 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:06.981674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33512 (local address 127.15.110.253:35871)
0927 18:48:06.981743 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:06.981745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:06.993472 (+ 11727us) server_negotiation.cc:407] Connection header received
0927 18:48:06.993604 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:06.993606 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:06.993620 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:06.993658 (+ 38us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:06.993900 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.994736 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.995905 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:06.996199 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:06.996237 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:06.996425 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:06.996460 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:06.996484 (+ 24us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:06.996577 (+ 93us) server_negotiation.cc:299] Negotiation successful
0927 18:48:06.996595 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:48:07.006182 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:07.003407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48822 (local address 127.15.110.252:40675)
0927 18:48:07.003452 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.003454 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.005469 (+ 2015us) server_negotiation.cc:407] Connection header received
0927 18:48:07.005665 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.005666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.005680 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.005709 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:07.006118 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48822: BlockingRecv error: recv got EOF from 127.0.0.1:48822 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
I20260927 18:48:07.048897 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:07.046196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.15.110.253:35871)
0927 18:48:07.046249 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.046251 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.046398 (+ 147us) server_negotiation.cc:407] Connection header received
0927 18:48:07.046475 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.046476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.046487 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.046539 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:07.046677 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.047336 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.048169 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.048383 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.048477 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:07.048638 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:07.048675 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:07.048727 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:07.048788 (+ 61us) server_negotiation.cc:299] Negotiation successful
0927 18:48:07.048808 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
W20260927 18:48:07.051425 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:07.050829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48834 (local address 127.15.110.252:40675)
0927 18:48:07.050881 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.050883 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.050892 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:07.050958 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.050958 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.050970 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.051015 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:07.051370 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48834: BlockingRecv error: recv got EOF from 127.0.0.1:48834 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260927 18:48:07.119050 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:07.116765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.15.110.253:35871)
0927 18:48:07.116860 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.116863 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.116964 (+ 101us) server_negotiation.cc:407] Connection header received
0927 18:48:07.117010 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.117011 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.117020 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.117072 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:07.117187 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.117651 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.118445 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.118619 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.118668 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:07.118812 (+ 144us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:07.118837 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:07.118880 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:07.118929 (+ 49us) server_negotiation.cc:299] Negotiation successful
0927 18:48:07.118943 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260927 18:48:07.121663 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:07.121044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48840 (local address 127.15.110.252:40675)
0927 18:48:07.121110 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.121112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.121150 (+ 38us) server_negotiation.cc:407] Connection header received
0927 18:48:07.121227 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.121229 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.121240 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.121295 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:07.121601 (+ 306us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48840: BlockingRecv error: recv got EOF from 127.0.0.1:48840 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260927 18:48:07.253575 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:07.251017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33554 (local address 127.15.110.253:35871)
0927 18:48:07.251100 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.251109 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.251216 (+ 107us) server_negotiation.cc:407] Connection header received
0927 18:48:07.251328 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.251329 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.251341 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.251415 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:07.251570 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.252082 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.252859 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.253060 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.253139 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:07.253291 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:07.253329 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:07.253390 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:07.253460 (+ 70us) server_negotiation.cc:299] Negotiation successful
0927 18:48:07.253479 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260927 18:48:07.255914 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:07.255378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48852 (local address 127.15.110.252:40675)
0927 18:48:07.255458 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.255460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.255468 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:07.255534 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.255535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.255544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.255589 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:07.255863 (+ 274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48852: BlockingRecv error: recv got EOF from 127.0.0.1:48852 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260927 18:48:07.459450 5795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6c6d7bf49c824976a18245a2e4d74ded" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c6d7bf49c824976a18245a2e4d74ded" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 46861 } } peers { permanent_uuid: "977d13757d144bcb89f3454fc572b56f" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 35871 } } peers { permanent_uuid: "2c1455720d7c49efb867677448c19a8b" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 40675 } } }
I20260927 18:48:07.459600 5795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6c6d7bf49c824976a18245a2e4d74ded [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:07.515305 5689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:07.513153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33570 (local address 127.15.110.253:35871)
0927 18:48:07.513219 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.513221 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.513231 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:48:07.513290 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.513291 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.513299 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.513359 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:07.513492 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.513969 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.514708 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:07.514878 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:07.514940 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:07.515092 (+ 152us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:07.515118 (+ 26us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:07.515155 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:07.515227 (+ 72us) server_negotiation.cc:299] Negotiation successful
0927 18:48:07.515243 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260927 18:48:07.517759 5763 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:07.517166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48868 (local address 127.15.110.252:40675)
0927 18:48:07.517241 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:07.517243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:07.517252 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:07.517279 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:07.517280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:07.517291 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:07.517364 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:07.517694 (+ 330us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48868: BlockingRecv error: recv got EOF from 127.0.0.1:48868 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260927 18:48:07.795275 5776 catalog_manager.cc:1259] Loaded cluster ID: 6e8ddaaea2d646318271b34f92f71c78
I20260927 18:48:07.795377 5776 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: loading cluster ID for follower catalog manager: success
I20260927 18:48:07.796195 5776 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: acquiring CA information for follower catalog manager: success
I20260927 18:48:07.796517 5776 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2c1455720d7c49efb867677448c19a8b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:48:08.033625 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:08.031067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33578 (local address 127.15.110.253:35871)
0927 18:48:08.031234 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:08.031237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:08.031248 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:48:08.031322 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:08.031324 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:08.031341 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:08.031406 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:08.031598 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.032106 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.032894 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.033101 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.033179 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:08.033333 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:08.033371 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:08.033430 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:08.033496 (+ 66us) server_negotiation.cc:299] Negotiation successful
0927 18:48:08.033523 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":54,"threads_started":1}
I20260927 18:48:08.037530 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:08.035257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.15.110.252:40675)
0927 18:48:08.035362 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:08.035363 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:08.035372 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:08.035396 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:08.035397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:08.035446 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:08.035515 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:08.035620 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.036091 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.036773 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.036999 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.037052 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:08.037185 (+ 133us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:08.037288 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:08.037350 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:08.037382 (+ 32us) server_negotiation.cc:299] Negotiation successful
0927 18:48:08.037406 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":54,"threads_started":1}
I20260927 18:48:08.042105 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:08.039226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52408 (local address 127.15.110.254:46861)
0927 18:48:08.039432 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:08.039435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:08.039452 (+ 17us) server_negotiation.cc:407] Connection header received
0927 18:48:08.039499 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:08.039500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:08.039555 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:08.039630 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:08.039779 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.040547 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.041299 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.041537 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.041609 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:08.041763 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:08.041884 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:08.041945 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:08.041997 (+ 52us) server_negotiation.cc:299] Negotiation successful
0927 18:48:08.042020 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":47,"threads_started":1}
I20260927 18:48:08.042346 5950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:08.039443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48886 (local address 127.15.110.252:40675)
0927 18:48:08.039511 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:08.039512 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:08.039749 (+ 237us) server_negotiation.cc:407] Connection header received
0927 18:48:08.039832 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:08.039832 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:08.039842 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:08.039894 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:08.039980 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.040443 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.041560 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.041838 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.041905 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:08.042080 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:08.042117 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:08.042147 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:08.042260 (+ 113us) server_negotiation.cc:299] Negotiation successful
0927 18:48:08.042280 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260927 18:48:08.042743 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:08.039321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33588 (local address 127.15.110.253:35871)
0927 18:48:08.039357 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:08.039359 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:08.039370 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:48:08.039498 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:08.039499 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:08.039508 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:08.039556 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:08.039675 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.040423 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.041944 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:08.042225 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:08.042288 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:08.042458 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:08.042490 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:08.042541 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:08.042635 (+ 94us) server_negotiation.cc:299] Negotiation successful
0927 18:48:08.042657 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:08.043957 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5541
I20260927 18:48:08.049686 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5616
I20260927 18:48:08.054939 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5691
2026-09-27T18:48:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2881 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 27 18:48:08 dist-test-slave-vs05 krb5kdc[5961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 27 18:48:08 dist-test-slave-vs05 krb5kdc[5961](info): set up 2 sockets
Sep 27 18:48:08 dist-test-slave-vs05 krb5kdc[5961](info): commencing operation
krb5kdc: starting...
W20260927 18:48:09.147545 15803 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/test-user.keytab.
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-27T18:48:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-27T18:48:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.15.110.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
Entry for principal kudu/127.15.110.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab.
I20260927 18:48:09.265250 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36397
--webserver_interface=127.15.110.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260927 18:48:09.346894 5974 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:09.347164 5974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:09.347198 5974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:09.347304 5974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:09.348503 5974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:09.348579 5974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:09.348603 5974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:09.348623 5974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:09.348654 5974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:09.350150 5974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.254.keytab
--principal=kudu/127.15.110.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.254:36397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.15.110.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:09.350447 5974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:09.350682 5974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:09.355700 5979 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:09.355692 5982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.355756 5974 server_base.cc:1034] running on GCE node
W20260927 18:48:09.355692 5980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.356184 5974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:09.356464 5974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:09.357613 5974 hybrid_clock.cc:648] HybridClock initialized: now 1790534889357589 us; error 40 us; skew 500 ppm
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:09.368078 5974 init.cc:377] Logged in from keytab as kudu/127.15.110.254@KRBTEST.COM (short username kudu)
I20260927 18:48:09.368649 5974 webserver.cc:460] Webserver started at http://127.15.110.254:35367/ using document root <none> and password file <none>
I20260927 18:48:09.368888 5974 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:09.368940 5974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:09.369079 5974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:09.370196 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data/instance:
uuid: "89163455f221445b84407a6f9ced3693"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "7fb8d2e37b75c9063e8e824ff42ed3c3"
server_key_iv: "1347d251e51ed31fd8fdc9442f3a3419"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.370570 5974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance:
uuid: "89163455f221445b84407a6f9ced3693"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "7fb8d2e37b75c9063e8e824ff42ed3c3"
server_key_iv: "1347d251e51ed31fd8fdc9442f3a3419"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.372059 5974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260927 18:48:09.372869 5989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:09.373066 5974 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260927 18:48:09.373168 5974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal
uuid: "89163455f221445b84407a6f9ced3693"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "7fb8d2e37b75c9063e8e824ff42ed3c3"
server_key_iv: "1347d251e51ed31fd8fdc9442f3a3419"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.373255 5974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:09.384570 5974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:09.386734 5974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:09.386904 5974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:09.391081 5974 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.254:36397
I20260927 18:48:09.391117 6041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.254:36397 every 8 connection(s)
I20260927 18:48:09.391522 5974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/data/info.pb
I20260927 18:48:09.392405 6042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:09.394109 6042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:09.400208 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 5974
I20260927 18:48:09.400319 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-0/wal/instance
I20260927 18:48:09.400496 15803 external_mini_cluster.cc:1428] Setting key 5592f8c9515fe32c14a4a865de04f9e9
I20260927 18:48:09.404088 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.394766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36397 (local address 127.0.0.1:59336)
0927 18:48:09.395016 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.395046 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.395557 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.395812 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.395814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.396080 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.396236 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.396240 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.397190 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.397191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.397845 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.397847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.398346 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.398347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.398410 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.398974 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.398982 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.400580 (+ 1598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.402490 (+ 1910us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.402497 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.402499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.402796 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.403048 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.403049 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.403050 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.403149 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.403651 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.403657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.403731 (+ 74us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.403804 (+ 73us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.403928 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":89,"threads_started":1}
I20260927 18:48:09.404202 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.394822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59336 (local address 127.15.110.254:36397)
0927 18:48:09.394963 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.394971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.395131 (+ 160us) server_negotiation.cc:407] Connection header received
0927 18:48:09.395688 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.395690 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.395706 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.395783 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.396319 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.397112 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.397966 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.398269 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.398346 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.400680 (+ 2334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.400688 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.400688 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.400695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.402402 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.402895 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.402898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.402898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.402978 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.403278 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.403279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.403280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.403581 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.403667 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.403883 (+ 216us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.404018 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260927 18:48:09.405072 6042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:09.405402 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.405300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51858)
0927 18:48:09.405335 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.405380 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260927 18:48:09.405587 5993 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
I20260927 18:48:09.406055 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.405956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51870)
0927 18:48:09.406007 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.406035 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260927 18:48:09.406195 6042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:32947: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.15.110.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
Entry for principal kudu/127.15.110.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab.
I20260927 18:48:09.426012 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:32947
--webserver_interface=127.15.110.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:48:09.455463 6042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } attempt: 1
I20260927 18:48:09.455996 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.455863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51878)
0927 18:48:09.455926 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.455968 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:48:09.456373 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.456294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51892)
0927 18:48:09.456333 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.456358 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260927 18:48:09.456478 6042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:32947: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
W20260927 18:48:09.507037 6049 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:09.507273 6049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:09.507290 6049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:09.507354 6049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:09.508546 6049 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:09.508587 6049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:09.508601 6049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:09.508646 6049 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:09.508677 6049 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:09.510136 6049 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.253.keytab
--principal=kudu/127.15.110.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.253:32947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.15.110.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:09.510401 6049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:09.510650 6049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:09.515470 6055 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:09.515662 6054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.515637 6049 server_base.cc:1034] running on GCE node
W20260927 18:48:09.515676 6057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.515950 6049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:09.516191 6049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:09.517329 6049 hybrid_clock.cc:648] HybridClock initialized: now 1790534889517303 us; error 38 us; skew 500 ppm
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:09.519775 6049 init.cc:377] Logged in from keytab as kudu/127.15.110.253@KRBTEST.COM (short username kudu)
I20260927 18:48:09.520218 6049 webserver.cc:460] Webserver started at http://127.15.110.253:42663/ using document root <none> and password file <none>
I20260927 18:48:09.520409 6049 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:09.520453 6049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:09.520538 6049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:09.521515 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data/instance:
uuid: "01090bd946e545d19f7eee6317247aec"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "ef506b847d4b1112ab5781653900ccb4"
server_key_iv: "c9f7b22087a55da1735c886de5fe323d"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.521804 6049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal/instance:
uuid: "01090bd946e545d19f7eee6317247aec"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "ef506b847d4b1112ab5781653900ccb4"
server_key_iv: "c9f7b22087a55da1735c886de5fe323d"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.523067 6049 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260927 18:48:09.523826 6064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:09.524015 6049 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:09.524083 6049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal
uuid: "01090bd946e545d19f7eee6317247aec"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "ef506b847d4b1112ab5781653900ccb4"
server_key_iv: "c9f7b22087a55da1735c886de5fe323d"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.524174 6049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:09.524631 6042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } attempt: 2
I20260927 18:48:09.525012 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.524890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51906)
0927 18:48:09.524944 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.524975 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260927 18:48:09.525386 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.525307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51914)
0927 18:48:09.525347 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.525370 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260927 18:48:09.525488 6042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.253:32947: Network error: Client connection negotiation failed: client connection to 127.15.110.253:32947: connect: Connection refused (error 111)
I20260927 18:48:09.536391 6049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:09.538324 6049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:09.538491 6049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:09.542503 6049 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.253:32947
I20260927 18:48:09.542533 6116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.253:32947 every 8 connection(s)
I20260927 18:48:09.542924 6049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/data/info.pb
I20260927 18:48:09.543790 6117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:09.545301 6117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:09.550647 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 6049
I20260927 18:48:09.550765 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-1/wal/instance
I20260927 18:48:09.550951 15803 external_mini_cluster.cc:1428] Setting key c57a41ae57613b38817dab4f132ae69e
I20260927 18:48:09.554472 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.546023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59342 (local address 127.15.110.254:36397)
0927 18:48:09.546095 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.546097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.546264 (+ 167us) server_negotiation.cc:407] Connection header received
0927 18:48:09.546705 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.546706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.546720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.546772 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.547435 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.547901 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.548828 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.549030 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.549084 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.551269 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.551270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.551271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.551284 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.552983 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.553432 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.553433 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.553434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.553499 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.553808 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.553809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.553809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.553938 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.554017 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.554291 (+ 274us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.554379 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260927 18:48:09.554692 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.545996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36397 (local address 127.0.0.1:59342)
0927 18:48:09.546149 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.546184 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.546624 (+ 440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.546842 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.546845 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.547093 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.547361 (+ 268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.547366 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.547998 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.547999 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.548754 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.548756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.549102 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.549103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.549205 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.549716 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.549724 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.551138 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.553072 (+ 1934us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.553079 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.553081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.553326 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.553571 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.553572 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.553573 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.553719 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.554008 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.554015 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.554159 (+ 144us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.554272 (+ 113us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.554437 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":76,"threads_started":1}
I20260927 18:48:09.555300 6117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.15.110.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.15.110.252@KRBTEST.COM" created.
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:09.564316 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.555587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51916)
0927 18:48:09.555650 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.555660 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.555699 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.556004 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.556005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.556145 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.556189 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.556191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.557024 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.557024 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.557523 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.557525 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.558029 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.558030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.558093 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.558526 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.558531 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.561432 (+ 2901us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.563217 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.563218 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.563219 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.563389 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.563617 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.563618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.563618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.563693 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.564084 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.564085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.564124 (+ 39us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.564180 (+ 56us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.564250 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260927 18:48:09.564560 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.555658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51916 (local address 127.15.110.253:32947)
0927 18:48:09.555871 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.555875 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.555890 (+ 15us) server_negotiation.cc:407] Connection header received
0927 18:48:09.555906 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.555908 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.555918 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.556005 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.556261 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.556926 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.557611 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.557940 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.558030 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.561534 (+ 3504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.561542 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.561542 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.561549 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.563109 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.563494 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.563501 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.563501 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.563549 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.563787 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.563788 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.563788 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.564004 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.564104 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.564244 (+ 140us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.564331 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260927 18:48:09.565248 6117 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:09.565610 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.565503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60016)
0927 18:48:09.565551 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.565593 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260927 18:48:09.565747 6068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
I20260927 18:48:09.566129 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.566042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60032)
0927 18:48:09.566085 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.566109 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:48:09.566234 6117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36429: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
Entry for principal kudu/127.15.110.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab.
I20260927 18:48:09.575889 15803 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskCruPYV/build/release/bin/kudu
/tmp/dist-test-taskCruPYV/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36429
--webserver_interface=127.15.110.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.15.110.212:40195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260927 18:48:09.615490 6117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } attempt: 1
I20260927 18:48:09.616042 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.615888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60036)
0927 18:48:09.615955 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.616005 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260927 18:48:09.616472 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.616380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60048)
0927 18:48:09.616423 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.616450 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:48:09.616600 6117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36429: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
I20260927 18:48:09.616608 6042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } attempt: 3
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:09.623687 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.616922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51918)
0927 18:48:09.616962 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.616973 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.617024 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.617186 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.617187 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.617346 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.617424 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.617426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.618099 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.618100 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.618660 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.618662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.619089 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.619090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.619169 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.619685 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.619691 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.621071 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.622593 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.622595 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.622595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.622777 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.623001 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.623002 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.623003 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.623066 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.623351 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.623352 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.623406 (+ 54us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.623489 (+ 83us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.623590 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260927 18:48:09.623685 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.616952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51918 (local address 127.15.110.253:32947)
0927 18:48:09.616994 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.616996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.617009 (+ 13us) server_negotiation.cc:407] Connection header received
0927 18:48:09.617111 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.617111 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.617122 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.617180 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.617505 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.618012 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.618749 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.618997 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.619087 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.621168 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.621170 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.621170 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.621177 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.622509 (+ 1332us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.622861 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.622863 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.622863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.622933 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.623128 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.623129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.623130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.623280 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.623364 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.623515 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.623590 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260927 18:48:09.624066 6042 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:09.624413 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.624302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60060)
0927 18:48:09.624351 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.624378 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260927 18:48:09.624740 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.624677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60072)
0927 18:48:09.624708 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.624728 (+ 20us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260927 18:48:09.624833 6042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36429: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
I20260927 18:48:09.655999 6042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } attempt: 1
I20260927 18:48:09.656515 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.656369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60082)
0927 18:48:09.656442 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.656486 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260927 18:48:09.657009 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.656872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60094)
0927 18:48:09.656952 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.656984 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260927 18:48:09.657148 6042 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36429: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
W20260927 18:48:09.660567 6124 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260927 18:48:09.660764 6124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260927 18:48:09.660781 6124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260927 18:48:09.660801 6124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260927 18:48:09.661901 6124 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260927 18:48:09.661936 6124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260927 18:48:09.661952 6124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260927 18:48:09.661963 6124 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260927 18:48:09.661976 6124 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260927 18:48:09.663450 6124 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.15.110.212:40195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.15.110.254:36397,127.15.110.253:32947,127.15.110.252:36429
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/krb5kdc/kudu_127.15.110.252.keytab
--principal=kudu/127.15.110.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.15.110.252:36429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.15.110.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6124
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 27 Sep 2026 17:43:27 UTC on ccycloud.quasar-iwwbht.root.comops.site
build id 4676275
I20260927 18:48:09.663643 6124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260927 18:48:09.663851 6124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260927 18:48:09.668655 6132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260927 18:48:09.668718 6130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.668798 6124 server_base.cc:1034] running on GCE node
W20260927 18:48:09.668655 6129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260927 18:48:09.669118 6124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260927 18:48:09.669350 6124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260927 18:48:09.670490 6124 hybrid_clock.cc:648] HybridClock initialized: now 1790534889670477 us; error 28 us; skew 500 ppm
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260927 18:48:09.673012 6124 init.cc:377] Logged in from keytab as kudu/127.15.110.252@KRBTEST.COM (short username kudu)
I20260927 18:48:09.673485 6124 webserver.cc:460] Webserver started at http://127.15.110.252:37541/ using document root <none> and password file <none>
I20260927 18:48:09.673722 6124 fs_manager.cc:359] Metadata directory not provided
I20260927 18:48:09.673789 6124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260927 18:48:09.673941 6124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260927 18:48:09.675021 6124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data/instance:
uuid: "461ef26b255543b79bc954cb3f31f30c"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "a156912c4f4186bc96782bd4ca87f816"
server_key_iv: "851e72716d896429cc4824d9af291312"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.675474 6124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal/instance:
uuid: "461ef26b255543b79bc954cb3f31f30c"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "a156912c4f4186bc96782bd4ca87f816"
server_key_iv: "851e72716d896429cc4824d9af291312"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.676868 6124 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260927 18:48:09.677606 6139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260927 18:48:09.677778 6124 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260927 18:48:09.677881 6124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data,/tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal
uuid: "461ef26b255543b79bc954cb3f31f30c"
format_stamp: "Formatted at 2026-09-27 18:48:09 on dist-test-slave-vs05"
server_key: "a156912c4f4186bc96782bd4ca87f816"
server_key_iv: "851e72716d896429cc4824d9af291312"
server_key_version: "encryptionkey@0"
I20260927 18:48:09.677955 6124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260927 18:48:09.684729 6117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } attempt: 2
I20260927 18:48:09.685164 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.685041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60096)
0927 18:48:09.685097 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.685136 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260927 18:48:09.685510 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.685415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60112)
0927 18:48:09.685459 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.685486 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260927 18:48:09.685624 6117 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.15.110.252:36429: Network error: Client connection negotiation failed: client connection to 127.15.110.252:36429: connect: Connection refused (error 111)
I20260927 18:48:09.691311 6124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260927 18:48:09.693617 6124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260927 18:48:09.693732 6124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260927 18:48:09.697793 6124 rpc_server.cc:307] RPC server started. Bound to: 127.15.110.252:36429
I20260927 18:48:09.697822 6191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.15.110.252:36429 every 8 connection(s)
I20260927 18:48:09.698181 6124 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/data/info.pb
I20260927 18:48:09.699018 6192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260927 18:48:09.700512 15803 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskCruPYV/build/release/bin/kudu as pid 6124
I20260927 18:48:09.700603 15803 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskCruPYV/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790534620099873-15803-0/minicluster-data/master-2/wal/instance
I20260927 18:48:09.700583 6192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } has no permanent_uuid. Determining permanent_uuid...
I20260927 18:48:09.700815 15803 external_mini_cluster.cc:1428] Setting key 8b7cbb06656bac96bc5201fee0add23c
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:09.710443 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.701553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59354 (local address 127.15.110.254:36397)
0927 18:48:09.701608 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.701610 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.701619 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:09.702081 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.702082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.702095 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.702150 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.702909 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.703672 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.704827 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.705061 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.705127 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.707340 (+ 2213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.707341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.707341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.707347 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.708976 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.709357 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.709359 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.709359 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.709416 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.709680 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.709681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.709681 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.709832 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.709926 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.710246 (+ 320us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.710328 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260927 18:48:09.710824 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.701314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.254:36397 (local address 127.0.0.1:59354)
0927 18:48:09.701494 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.701533 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.701995 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.702182 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.702185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.702568 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.702822 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.702827 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.703745 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.703746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.704742 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.704745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.705142 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.705143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.705268 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.705774 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.705784 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.707261 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.709066 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.709072 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.709073 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.709286 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.709495 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.709495 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.709496 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.709589 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.709932 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.709938 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.710099 (+ 161us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.710211 (+ 112us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.710327 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":1,"thread_start_us":95,"threads_started":1}
I20260927 18:48:09.710830 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.701311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60118 (local address 127.15.110.252:36429)
0927 18:48:09.701519 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.701529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.701557 (+ 28us) server_negotiation.cc:407] Connection header received
0927 18:48:09.702012 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.702024 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.702067 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.702163 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.702910 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.704028 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.704996 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.705393 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.705481 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.707990 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.707998 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.707998 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.708005 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.709573 (+ 1568us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.709905 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.709908 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.709908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.709966 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.710168 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.710169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.710169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.710362 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.710512 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.710623 (+ 111us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.710687 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":166,"thread_start_us":96,"threads_started":1}
I20260927 18:48:09.711421 6192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.254@KRBTEST.COM
I20260927 18:48:09.717685 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.711739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51932 (local address 127.15.110.253:32947)
0927 18:48:09.711782 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.711784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.711807 (+ 23us) server_negotiation.cc:407] Connection header received
0927 18:48:09.711832 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.711833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.711843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.711907 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.712167 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.712635 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.713270 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.713491 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.713577 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.715345 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.715346 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.715347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.715353 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.716660 (+ 1307us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.716981 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.716983 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.716983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.717039 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.717247 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.717247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.717248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.717357 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.717430 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.717569 (+ 139us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.717625 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:48:09.717742 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.711685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.253:32947 (local address 127.0.0.1:51932)
0927 18:48:09.711741 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.711751 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.711781 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.711905 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.711906 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.712048 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.712081 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.712082 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.712730 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.712731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.713192 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.713193 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.713577 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.713578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.713638 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.714037 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.714042 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.715259 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.716754 (+ 1495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.716755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.716756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.716909 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.717121 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.717121 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.717122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.717176 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.717429 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.717430 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.717473 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.717544 (+ 71us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.717629 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260927 18:48:09.718585 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.711679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59356 (local address 127.15.110.254:36397)
0927 18:48:09.711713 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.711715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.711722 (+ 7us) server_negotiation.cc:407] Connection header received
0927 18:48:09.711742 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.711742 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.711755 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.711802 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.712372 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.712820 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.713488 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.713688 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.713775 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.716060 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.716061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.716061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.716068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.717431 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.717783 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.717784 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.717785 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.717838 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.718032 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.718033 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.718033 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.718161 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.718364 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.718405 (+ 41us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.718476 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:48:09.718581 6192 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } has no permanent_uuid. Determining permanent_uuid...
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.252@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.15.110.253@KRBTEST.COM
I20260927 18:48:09.725620 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.718874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60132 (local address 127.15.110.252:36429)
0927 18:48:09.718912 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.718914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.718922 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:09.719001 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.719002 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.719011 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.719081 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.719399 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.719854 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.720503 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.720711 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.720795 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.722779 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.722780 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.722780 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.722786 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.724444 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.724803 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.724805 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.724805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.724856 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.725052 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.725053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.725054 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.725182 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.725261 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.725389 (+ 128us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.725513 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":56,"server-negotiator.queue_time_us":19}
I20260927 18:48:09.725632 6195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.718844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60132)
0927 18:48:09.718884 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.718894 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.718927 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.719092 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.719093 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.719285 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.719315 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.719317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.719943 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.719944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.720405 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.720406 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.720794 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.720795 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.720841 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.721244 (+ 403us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.721249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.722613 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.724536 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.724537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.724537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.724731 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.724938 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.724939 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.724939 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.724990 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.725256 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.725256 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.725288 (+ 32us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.725366 (+ 78us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.725466 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260927 18:48:09.726683 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.719243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51938 (local address 127.15.110.253:32947)
0927 18:48:09.719283 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.719285 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.719293 (+ 8us) server_negotiation.cc:407] Connection header received
0927 18:48:09.719345 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.719346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.719355 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.719401 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.719860 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.720293 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.720956 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.721186 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.721263 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.723773 (+ 2510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.723774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.723774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.723781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.725286 (+ 1505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.725670 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.725671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.725671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.725795 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.726163 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.726164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.726164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.726281 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.726483 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.726520 (+ 37us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.726590 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:48:09.727283 6192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: Bootstrap starting.
I20260927 18:48:09.728102 15803 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260927 18:48:09.728245 6192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:09.728473 6192 log.cc:824] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:09.729310 6192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: No bootstrap required, opened a new log
I20260927 18:48:09.729123 6016 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "89163455f221445b84407a6f9ced3693"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59356
I20260927 18:48:09.731390 6192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.731552 6192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 461ef26b255543b79bc954cb3f31f30c, State: Initialized, Role: FOLLOWER
I20260927 18:48:09.731630 6192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.731941 6192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:09.731885 6198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.732012 6198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.732307 6042 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } attempt: 2
W20260927 18:48:09.733929 6209 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:09.733985 6209 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.254@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:09.734944 6124 master_runner.cc:186] Error getting master registration for 127.15.110.254:36397: OK, instance_id { permanent_uuid: "89163455f221445b84407a6f9ced3693" instance_seqno: 1790534889389899 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:48:09.737388 6124 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:48:09.740295 6044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.732562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60140)
0927 18:48:09.732614 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.732627 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.732683 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.732809 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.732826 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.733032 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.733079 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.733082 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.733939 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.733940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.734715 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.734718 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.735239 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.735240 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.735299 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.735758 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.735764 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.737304 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.739083 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.739084 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.739084 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.739386 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.739613 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.739614 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.739614 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.739679 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.739990 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.739991 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.740034 (+ 43us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.740101 (+ 67us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.740194 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260927 18:48:09.740304 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.732640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60140 (local address 127.15.110.252:36429)
0927 18:48:09.732677 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.732678 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.732687 (+ 9us) server_negotiation.cc:407] Connection header received
0927 18:48:09.732748 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.732749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.732756 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.732796 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.733147 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.733854 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.734815 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.735112 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.735212 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.737400 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.737402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.737403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.737409 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.738984 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.739475 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.739476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.739477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.739537 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.739745 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.739746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.739747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.739905 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.739993 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.740144 (+ 151us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.740221 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:48:09.742028 6042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: Bootstrap starting.
I20260927 18:48:09.742811 6042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:09.743124 6042 log.cc:824] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:09.744005 6042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: No bootstrap required, opened a new log
I20260927 18:48:09.745371 6042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.745563 6042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89163455f221445b84407a6f9ced3693, State: Initialized, Role: FOLLOWER
I20260927 18:48:09.745666 6042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.746002 6042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:09.746032 6016 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260927 18:48:09.746095 6016 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:09.746788 6016 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.747035 6016 leader_election.cc:288] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [CANDIDATE]: Term 1 election: Requested vote from peers 01090bd946e545d19f7eee6317247aec (127.15.110.253:32947), 461ef26b255543b79bc954cb3f31f30c (127.15.110.252:36429)
I20260927 18:48:09.747364 6166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89163455f221445b84407a6f9ced3693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "461ef26b255543b79bc954cb3f31f30c"
I20260927 18:48:09.747499 6166 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:09.748421 6166 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89163455f221445b84407a6f9ced3693 in term 1.
I20260927 18:48:09.748637 5993 leader_election.cc:302] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 461ef26b255543b79bc954cb3f31f30c, 89163455f221445b84407a6f9ced3693; no voters:
I20260927 18:48:09.745970 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.748909 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.748970 6212 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 1 FOLLOWER]: Leader election won for term 1
I20260927 18:48:09.749032 6212 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [term 1 LEADER]: Becoming Leader. State: Replica: 89163455f221445b84407a6f9ced3693, State: Running, Role: LEADER
I20260927 18:48:09.749109 6212 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
W20260927 18:48:09.749576 6227 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:09.749655 6227 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:09.749526 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89163455f221445b84407a6f9ced3693. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.749998 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:09.751528 6232 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260927 18:48:09.751704 6232 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260927 18:48:09.749512 5974 master_runner.cc:186] Error getting master registration for 127.15.110.253:32947: OK, instance_id { permanent_uuid: "01090bd946e545d19f7eee6317247aec" instance_seqno: 1790534889541416 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260927 18:48:09.751858 5974 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260927 18:48:09.753472 6166 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [term 1 FOLLOWER]: Refusing update from remote peer 89163455f221445b84407a6f9ced3693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:48:09.753829 6212 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:48:09.755141 6198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89163455f221445b84407a6f9ced3693. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.755255 6198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.761878 6235 mvcc.cc:205] Tried to move back new op lower bound from 7334030908427157504 to 7334030908413063168. Current Snapshot: MvccSnapshot[applied={T|T < 7334030908427157504}]
I20260927 18:48:09.762446 6198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.762801 6198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.763410 6212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.763590 6212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:09.763702 6236 mvcc.cc:205] Tried to move back new op lower bound from 7334030908427157504 to 7334030908413063168. Current Snapshot: MvccSnapshot[applied={T|T < 7334030908427157504}]
I20260927 18:48:09.766016 6121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.750285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51940 (local address 127.15.110.253:32947)
0927 18:48:09.750326 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.750328 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.752501 (+ 2173us) server_negotiation.cc:407] Connection header received
0927 18:48:09.753722 (+ 1221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.753723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.753735 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.753763 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.755965 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.756703 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.758688 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.759024 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.759096 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.762218 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.762220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.762220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.762228 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.764157 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.764878 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.764879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.764880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.764945 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.765293 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.765294 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.765295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.765433 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.765514 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.765838 (+ 324us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.765918 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260927 18:48:09.767568 6045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.749724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59364 (local address 127.15.110.254:36397)
0927 18:48:09.749760 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.749762 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.751444 (+ 1682us) server_negotiation.cc:407] Connection header received
0927 18:48:09.753620 (+ 2176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.753621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.753632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.753661 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.755077 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.755864 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.761183 (+ 5319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.761492 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.761531 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.763863 (+ 2332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.763864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.763864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.763871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.766069 (+ 2198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.766576 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.766577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.766577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.766631 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.766817 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.766818 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.766818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.766937 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.767033 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.767379 (+ 346us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.767467 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:09.768859 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.750590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60150 (local address 127.15.110.252:36429)
0927 18:48:09.750623 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.750625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.752581 (+ 1956us) server_negotiation.cc:407] Connection header received
0927 18:48:09.752765 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.752766 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.752776 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.752804 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.758723 (+ 5919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.759477 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.760520 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.760815 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.760853 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.763583 (+ 2730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.763584 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.763585 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.763591 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.765363 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.767744 (+ 2381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.767745 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.767746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.767804 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.768182 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.768183 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.768184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.768322 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.768396 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.768698 (+ 302us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.768774 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:48:09.769946 6232 catalog_manager.cc:1347] Generated new cluster ID: bf39806df885428c87213d929029cd75
I20260927 18:48:09.770041 6232 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260927 18:48:09.776749 6117 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } attempt: 3
Sep 27 18:48:09 dist-test-slave-vs05 krb5kdc[5961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790534889, etypes {rep=17 tkt=17 ses=17}, kudu/127.15.110.253@KRBTEST.COM for kudu/127.15.110.252@KRBTEST.COM
I20260927 18:48:09.785095 6119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.776990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.15.110.252:36429 (local address 127.0.0.1:60152)
0927 18:48:09.777073 (+ 83us) negotiation.cc:107] Waiting for socket to connect
0927 18:48:09.777084 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0927 18:48:09.777117 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0927 18:48:09.777241 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0927 18:48:09.777242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0927 18:48:09.777395 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0927 18:48:09.777438 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.777439 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.778299 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.778300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.779089 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0927 18:48:09.779090 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.779576 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.779577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0927 18:48:09.779626 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.780037 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0927 18:48:09.780041 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0927 18:48:09.781672 (+ 1631us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0927 18:48:09.783658 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.783659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.783659 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.783879 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.784342 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0927 18:48:09.784343 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0927 18:48:09.784343 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0927 18:48:09.784429 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0927 18:48:09.784772 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0927 18:48:09.784773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0927 18:48:09.784831 (+ 58us) client_negotiation.cc:769] Sending connection context
0927 18:48:09.784892 (+ 61us) client_negotiation.cc:240] Negotiation successful
0927 18:48:09.784989 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260927 18:48:09.785637 6196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:09.777089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60152 (local address 127.15.110.252:36429)
0927 18:48:09.777124 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.777126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.777136 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:48:09.777159 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.777160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.777169 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.777238 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.777507 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.778221 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.779204 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:09.779481 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:09.779531 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:09.781799 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0927 18:48:09.781800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0927 18:48:09.781800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0927 18:48:09.781807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0927 18:48:09.783547 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.783970 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.783971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.783971 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.784041 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0927 18:48:09.784513 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0927 18:48:09.784514 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0927 18:48:09.784515 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0927 18:48:09.784672 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0927 18:48:09.784730 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:09.785473 (+ 743us) server_negotiation.cc:299] Negotiation successful
0927 18:48:09.785547 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260927 18:48:09.787274 6117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: Bootstrap starting.
I20260927 18:48:09.787983 6117 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: Neither blocks nor log segments found. Creating new log.
I20260927 18:48:09.788249 6117 log.cc:824] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: Log is configured to *not* fsync() on all Append() calls
I20260927 18:48:09.789268 6117 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: No bootstrap required, opened a new log
I20260927 18:48:09.791396 6117 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.791525 6117 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01090bd946e545d19f7eee6317247aec, State: Initialized, Role: FOLLOWER
I20260927 18:48:09.791594 6117 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } }
I20260927 18:48:09.791872 6117 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: configured and running, proceeding with master startup.
I20260927 18:48:09.791985 6090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [term 0 FOLLOWER]: Advancing to term 1
I20260927 18:48:09.792938 6090 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [term 1 FOLLOWER]: Refusing update from remote peer 89163455f221445b84407a6f9ced3693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260927 18:48:09.791882 6241 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.793238 6212 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260927 18:48:09.793368 6241 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.792008 6091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "89163455f221445b84407a6f9ced3693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "01090bd946e545d19f7eee6317247aec"
I20260927 18:48:09.794615 6091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 89163455f221445b84407a6f9ced3693 in term 1.
I20260927 18:48:09.795886 6255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89163455f221445b84407a6f9ced3693. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.796030 6255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: This master's current role is: FOLLOWER
I20260927 18:48:09.796104 6255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.796193 6255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec [sys.catalog]: This master's current role is: FOLLOWER
W20260927 18:48:09.799644 6252 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260927 18:48:09.799698 6252 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260927 18:48:09.803670 6253 mvcc.cc:205] Tried to move back new op lower bound from 7334030908427157504 to 7334030908413063168. Current Snapshot: MvccSnapshot[applied={T|T < 7334030908427157504}]
I20260927 18:48:09.838059 6228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "89163455f221445b84407a6f9ced3693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "89163455f221445b84407a6f9ced3693" member_type: VOTER last_known_addr { host: "127.15.110.254" port: 36397 } } peers { permanent_uuid: "01090bd946e545d19f7eee6317247aec" member_type: VOTER last_known_addr { host: "127.15.110.253" port: 32947 } } peers { permanent_uuid: "461ef26b255543b79bc954cb3f31f30c" member_type: VOTER last_known_addr { host: "127.15.110.252" port: 36429 } } }
I20260927 18:48:09.838215 6228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693 [sys.catalog]: This master's current role is: LEADER
I20260927 18:48:09.845316 6232 catalog_manager.cc:1370] Generated new certificate authority record
I20260927 18:48:09.846141 6232 catalog_manager.cc:1504] Loading token signing keys...
I20260927 18:48:09.852882 6232 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89163455f221445b84407a6f9ced3693: Generated new TSK 0
I20260927 18:48:09.853075 6232 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260927 18:48:09.876075 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.874315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51946 (local address 127.15.110.253:32947)
0927 18:48:09.874363 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.874365 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.875292 (+ 927us) server_negotiation.cc:407] Connection header received
0927 18:48:09.875454 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.875456 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.875468 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.875501 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.875987 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51946: BlockingRecv error: recv got EOF from 127.0.0.1:51946 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:48:09.877065 6196 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.874726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60156 (local address 127.15.110.252:36429)
0927 18:48:09.874776 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.874779 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.876333 (+ 1554us) server_negotiation.cc:407] Connection header received
0927 18:48:09.876607 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.876608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.876619 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.876649 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.877004 (+ 355us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60156: BlockingRecv error: recv got EOF from 127.0.0.1:60156 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260927 18:48:09.877700 6045 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.874982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59370 (local address 127.15.110.254:36397)
0927 18:48:09.875021 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.875023 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.876556 (+ 1533us) server_negotiation.cc:407] Connection header received
0927 18:48:09.877277 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.877278 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.877289 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.877320 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.877638 (+ 318us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59370: BlockingRecv error: recv got EOF from 127.0.0.1:59370 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 18:48:09.880232 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.879370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51962 (local address 127.15.110.253:32947)
0927 18:48:09.879416 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.879430 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.879675 (+ 245us) server_negotiation.cc:407] Connection header received
0927 18:48:09.879783 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.879784 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.879803 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.879833 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.880171 (+ 338us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51962: BlockingRecv error: recv got EOF from 127.0.0.1:51962 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 18:48:09.883198 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.882380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.15.110.253:32947)
0927 18:48:09.882414 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.882416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.882671 (+ 255us) server_negotiation.cc:407] Connection header received
0927 18:48:09.882793 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.882794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.882805 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.882833 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.883134 (+ 301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51964: BlockingRecv error: recv got EOF from 127.0.0.1:51964 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260927 18:48:09.886958 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.886217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51970 (local address 127.15.110.253:32947)
0927 18:48:09.886250 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.886251 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.886501 (+ 250us) server_negotiation.cc:407] Connection header received
0927 18:48:09.886594 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.886595 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.886604 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.886631 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.886918 (+ 287us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51970: BlockingRecv error: recv got EOF from 127.0.0.1:51970 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260927 18:48:09.892858 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.892035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51976 (local address 127.15.110.253:32947)
0927 18:48:09.892071 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.892073 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.892324 (+ 251us) server_negotiation.cc:407] Connection header received
0927 18:48:09.892433 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.892434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.892444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.892473 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.892808 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51976: BlockingRecv error: recv got EOF from 127.0.0.1:51976 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260927 18:48:09.902905 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.901975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.15.110.253:32947)
0927 18:48:09.902014 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.902016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.902280 (+ 264us) server_negotiation.cc:407] Connection header received
0927 18:48:09.902384 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.902385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.902397 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.902440 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.902849 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51984: BlockingRecv error: recv got EOF from 127.0.0.1:51984 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260927 18:48:09.921366 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.920296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52000 (local address 127.15.110.253:32947)
0927 18:48:09.920340 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.920342 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.920665 (+ 323us) server_negotiation.cc:407] Connection header received
0927 18:48:09.920776 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.920777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.920790 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.920819 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.921305 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52000: BlockingRecv error: recv got EOF from 127.0.0.1:52000 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 18:48:09.955626 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:09.954781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.15.110.253:32947)
0927 18:48:09.954829 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:09.954831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:09.955027 (+ 196us) server_negotiation.cc:407] Connection header received
0927 18:48:09.955097 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:09.955098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:09.955109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:09.955159 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:09.955563 (+ 404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52002: BlockingRecv error: recv got EOF from 127.0.0.1:52002 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260927 18:48:10.021577 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:10.020949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52012 (local address 127.15.110.253:32947)
0927 18:48:10.021023 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.021025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.021036 (+ 11us) server_negotiation.cc:407] Connection header received
0927 18:48:10.021081 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.021081 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.021094 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.021164 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:10.021522 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52012: BlockingRecv error: recv got EOF from 127.0.0.1:52012 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260927 18:48:10.151515 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:10.150774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52026 (local address 127.15.110.253:32947)
0927 18:48:10.150845 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.150847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.150974 (+ 127us) server_negotiation.cc:407] Connection header received
0927 18:48:10.151036 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.151037 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.151048 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.151095 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:10.151453 (+ 358us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52026: BlockingRecv error: recv got EOF from 127.0.0.1:52026 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260927 18:48:10.409322 6121 negotiation.cc:337] Failed RPC negotiation. Trace:
0927 18:48:10.408777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.15.110.253:32947)
0927 18:48:10.408849 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.408851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.408861 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:48:10.408887 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.408888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.408898 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.408964 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0927 18:48:10.409275 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59558: BlockingRecv error: recv got EOF from 127.0.0.1:59558 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
I20260927 18:48:10.734436 6209 catalog_manager.cc:1259] Loaded cluster ID: bf39806df885428c87213d929029cd75
I20260927 18:48:10.734521 6209 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: loading cluster ID for follower catalog manager: success
I20260927 18:48:10.735488 6209 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: acquiring CA information for follower catalog manager: success
I20260927 18:48:10.735836 6209 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 461ef26b255543b79bc954cb3f31f30c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:48:10.800158 6252 catalog_manager.cc:1259] Loaded cluster ID: bf39806df885428c87213d929029cd75
I20260927 18:48:10.800253 6252 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: loading cluster ID for follower catalog manager: success
I20260927 18:48:10.801115 6252 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: acquiring CA information for follower catalog manager: success
I20260927 18:48:10.801472 6252 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 01090bd946e545d19f7eee6317247aec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260927 18:48:10.925222 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:10.922653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59570 (local address 127.15.110.253:32947)
0927 18:48:10.922768 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.922769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.922779 (+ 10us) server_negotiation.cc:407] Connection header received
0927 18:48:10.922861 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.922862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.922921 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.923007 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:10.923155 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.923661 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.924460 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.924653 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.924708 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:10.924871 (+ 163us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:10.924993 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:10.925051 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:10.925134 (+ 83us) server_negotiation.cc:299] Negotiation successful
0927 18:48:10.925154 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":67,"threads_started":1}
I20260927 18:48:10.929682 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:10.927271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56996 (local address 127.15.110.252:36429)
0927 18:48:10.927409 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.927411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.927464 (+ 53us) server_negotiation.cc:407] Connection header received
0927 18:48:10.927543 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.927544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.927578 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.927645 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:10.927781 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.928315 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.928987 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.929176 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.929229 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:10.929349 (+ 120us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:10.929450 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:10.929506 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:10.929541 (+ 35us) server_negotiation.cc:299] Negotiation successful
0927 18:48:10.929557 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":76,"threads_started":1}
I20260927 18:48:10.934900 6330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:10.931881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.15.110.252:36429)
0927 18:48:10.931917 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.931919 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.932008 (+ 89us) server_negotiation.cc:407] Connection header received
0927 18:48:10.932099 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.932099 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.932107 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.932162 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:10.932312 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.933021 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.934165 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.934365 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.934437 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:10.934593 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:10.934624 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:10.934672 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:10.934798 (+ 126us) server_negotiation.cc:299] Negotiation successful
0927 18:48:10.934820 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:10.935067 6324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:10.931569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.15.110.253:32947)
0927 18:48:10.931606 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.931608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.931862 (+ 254us) server_negotiation.cc:407] Connection header received
0927 18:48:10.931899 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.931900 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.931911 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.931968 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:10.932204 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.932996 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.934145 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.934380 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.934460 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:10.934624 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:10.934657 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:10.934710 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:10.935002 (+ 292us) server_negotiation.cc:299] Negotiation successful
0927 18:48:10.935021 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260927 18:48:10.935441 6337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0927 18:48:10.931552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58984 (local address 127.15.110.254:36397)
0927 18:48:10.931661 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0927 18:48:10.931663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0927 18:48:10.931710 (+ 47us) server_negotiation.cc:407] Connection header received
0927 18:48:10.931836 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0927 18:48:10.931837 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0927 18:48:10.931891 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0927 18:48:10.931982 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0927 18:48:10.932136 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.932808 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.933849 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0927 18:48:10.934074 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0927 18:48:10.934156 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0927 18:48:10.934656 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0927 18:48:10.934775 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0927 18:48:10.934830 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0927 18:48:10.935328 (+ 498us) server_negotiation.cc:299] Negotiation successful
0927 18:48:10.935354 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87,"thread_start_us":53,"threads_started":1}
I20260927 18:48:10.937074 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 5974
I20260927 18:48:10.943293 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 6049
I20260927 18:48:10.948376 15803 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskCruPYV/build/release/bin/kudu with pid 6124
2026-09-27T18:48:10Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2893 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5775 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (270856 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST